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Under  static  conditions,  a  system  satisfying  detailed  balance  genetically  relaxes  to  an  equilibrium  state  in 
which  there  are  no  currents.  To  generate  persistent  currents,  either  detailed  balance  must  be  broken  or  the 
system  must  be  driven  in  a  time-dependent  manner.  A  stationary  system  that  violates  detailed  balance 
evolves  to  a  nonequilibrium  steady  state  (NESS)  characterized  by  fixed  currents.  Conversely,  a  system  that 
satisfies  instantaneous  detailed  balance  but  is  driven  by  the  time-periodic  variation  of  external  parameters — 
also  known  as  a  stochastic  pump  (SP) — reaches  a  periodic  state  with  nonvanishing  currents.  In  both  cases, 
these  currents  are  maintained  at  the  cost  of  entropy  production.  Are  these  two  paradigmatic  scenarios 
effectively  equivalent?  For  discrete-state  systems,  we  establish  a  mapping  between  nonequilibrium 
stationary  states  and  stochastic  pumps.  Given  a  NESS  characterized  by  a  particular  set  of  stationary 
probabilities,  currents,  and  entropy  production  rates,  we  show  how  to  construct  a  SP  with  exactly  the  same 
(time-averaged)  values.  The  mapping  works  in  the  opposite  direction  as  well.  These  results  establish  a 
proof  of  principle:  They  show  that  stochastic  pumps  are  able  to  mimic  the  behavior  of  nonequilibrium 
steady  states,  and  vice  versa,  within  the  theoretical  framework  of  discrete-state  stochastic  thermodynamics. 
Nonequilibrium  steady  states  and  stochastic  pumps  are  often  used  to  model,  respectively,  biomolecular 
motors  driven  by  chemical  reactions  and  artificial  molecular  machines  steered  by  the  variation  of  external, 
macroscopic  parameters.  Our  results  loosely  suggest  that  anything  a  biomolecular  machine  can  do,  an 
artificial  molecular  machine  can  do  equally  well.  We  illustrate  this  principle  by  showing  that  kinetic 
proofreading,  a  NESS  mechanism  that  explains  the  low  error  rates  in  biochemical  reactions,  can  be 
effectively  mimicked  by  a  constrained  periodic  driving. 
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I.  INTRODUCTION  AND  MOTIVATION 

While  there  is  currently  no  single  theory  that  unifies  all 
nonequilibrium  phenomena,  a  number  of  useful  paradigms 
of  nonequilibrium  behavior  have  emerged.  These  include 
small  perturbations  near  equilibrium  [1],  systems  driven 
away  from  an  initial  state  of  equilibrium  [2],  spontaneous 
relaxation  towards  equilibrium  [3],  nonequilibrium  steady 
states  generated  by  fixed  thermodynamic  forces  [4], 
and  stochastic  pumps  driven  by  the  time -periodic  variation 
of  external  parameters  [5,6].  Theoretical  frameworks 
developed  within  each  paradigm — for  instance,  linear 
response  theory  to  describe  near-equilibrium  perturbations 
[7] — have  contributed  to  a  broader  understanding  of  non¬ 
equilibrium  processes. 
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In  this  work,  we  focus  on  two  of  these  paradigms: 
nonequilibrium  steady  states  and  stochastic  pumps. 
These  share  certain  features,  notably  the  persistence  of 
nonvanishing  currents  and  entropy  production,  which 
invite  a  comparison  between  the  two.  As  elaborated  below, 
we  devise  a  mapping  from  one  paradigm  to  the  other; 
given  a  system  in  a  nonequilibrium  steady  state  charac¬ 
terized  by  certain  occupation  probabilities,  currents,  and 
entropy  production  rates,  we  show  how  to  construct  a 
stochastic  pump  that  exhibits  the  same  (time-averaged) 
properties.  The  inverse  direction,  namely,  the  construction 
of  a  nonequilibrium  steady  state  with  the  same  properties 
as  a  given,  time-averaged  stochastic  pump,  will  also  be 
discussed. 

In  the  nonequilibrium  steady-state  (NESS)  paradigm,  a 
system  driven  by  fixed  thermodynamic  forces — such  as 
temperature  gradients  or  chemical  potential  differences — 
reaches  a  steady  state  in  which  its  statistical  properties  are 
stationary  with  time.  Unlike  an  equilibrium  state,  a  non¬ 
equilibrium  steady  state  exhibits  nonvanishing  currents, 
reflecting  the  violation  of  detailed  balance.  In  order  to 
maintain  such  a  state,  a  thermodynamic  cost  must  be  paid. 
This  cost  is  measured  by  the  continual  depletion  of  a 
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thermodynamic  resource,  such  as  a  chemical  fuel,  resulting 
in  the  production  of  entropy  in  the  system’s  thermal 
surroundings. 

Biomolecular  motors  illustrate  the  NESS  paradigm  [8,9]: 
A  reaction  such  as  ATP  hydrolysis  {ATP  — *■  ADP  +  P,) 
produces  entropy  in  the  surrounding  solution,  and  the 
chemical  potential  difference  between  reactants  and  prod¬ 
ucts  provides  the  thermodynamic  force.  The  “current”  in 
this  situation  corresponds  to  the  mechanical  motion  pro¬ 
duced  by  the  motor,  for  instance,  the  stepping  motion  of  the 
kinesin  motor  toward  the  positive  end  of  a  microtubule 
filament  [10,1 1],  or  the  rotary  motion  of  the  FI  domain  in 
ATP  synthase  [12,13].  For  a  recent  review  of  the  stochastic 
theory  of  nonequilibrium  steady  states,  as  applied  to 
biochemical  processes,  see  Ref.  [4]. 

In  the  stochastic  pump  paradigm,  a  system  is  driven  by  the 
time-periodic  variation  of  external  parameters  in  the  pres¬ 
ence  of  a  thermal  reservoir.  It  is  typically  assumed  that  the 
dynamics  satisfy  detailed  balance  at  every  instant  in  time — 
in  other  words,  if  the  parameters  were  suddenly  frozen  at 
their  instantaneous  values,  the  system  would  relax  to 
equilibrium.  Under  suitable  conditions,  a  periodically  driven 
system  reaches  a  time-periodic  state  with  nonvanishing 
time-averaged  currents.  These  currents  are  effectively 
“pumped”  by  the  periodic  variation  of  the  parameters, 
and  the  cost  associated  with  pumping  these  currents  is  the 
work  invested  in  driving  the  parameters.  Ultimately,  the 
energy  provided  by  this  work  is  dissipated  into  the  thermal 
reservoir,  resulting  in  the  production  of  entropy. 

The  study  of  stochastic  pumps  has  been  stimulated  by 
experiments  on  artificial  molecular  machines  [14-16], 
which  are  manipulated  by  the  variation  of  external  param¬ 
eters  to  achieve  some  desired  behavior.  For  instance,  in 
experiments  on  catenanes — mechanically  interlocked  ring¬ 
like  molecules — the  aim  was  to  produce  unidirectional 
rotation  of  one  ring  around  the  other  [17].  Theoretical 
investigations  of  SP  have  focused  on  slowly  driven  [5,18] 
as  well  as  weakly  driven  [19]  pumps,  “no-pumping” 
theorems  [20-24],  the  role  of  interactions  [25]  and  fluc¬ 
tuations  [26],  and  the  ability  to  extract  work  from  stochastic 
pumps  [27]. 

Underlying  both  the  experimental  work  on  artificial 
molecular  machines  and  the  theoretical  work  on  stochastic 
pumps  is  the  broad  goal  of  understanding  how  to  achieve 
controlled,  directed  motion  at  the  molecular  level.  The 
focus  on  time-dependent  driving  is  motivated  in  part  by  the 
difficulty  of  synthesizing  artificial  molecular  systems  that 
take  advantage  of  chemical  potential  differences  to  generate 
motion.  It  is  often  simpler  to  manipulate  the  system  by 
varying  external  parameters  such  as  temperature  and  the 
surrounding  chemical  environment.  From  a  theoretical 
perspective,  it  is  then  natural  to  investigate  the  range  of 
behaviors  that  can  be  achieved  when  driving  a  stochastic 
system  by  varying  its  parameters.  Biomolecular  motors 
driven  by  chemical  reactions  are  capable  of  accomplishing 


a  remarkably  diverse  array  of  tasks.  In  principle,  are 
artificial  molecular  machines  driven  by  time-periodic 
forcing  equally  versatile? 

In  both  nonequilibrium  steady  states  and  stochastic 
pumps,  the  generated  currents  can  be  viewed  as  desired 
outcomes,  and  entropy  production  is  the  cost  of  achieving 
them.  In  this  perspective,  fixed  thermodynamic  forces 
(NESS)  and  time-periodic  external  driving  (stochastic 
pump,  SP)  represent  the  tools  at  our  disposal.  We  compare 
these  two  sets  of  tools  with  respect  to  the  degree  of  control 
that  can  be  achieved.  In  particular,  we  investigate  whether 
time-periodic  driving  can  always  produce  the  same  out¬ 
come  (identical  currents)  as  fixed  thermodynamic  forces, 
and  at  the  same  cost  (identical  entropy  production),  after 
time  averaging.  In  other  words,  can  a  stochastic  pump 
“mimic”  an  arbitrary  nonequilibrium  steady  state? 

Somewhat  different  aspects  of  the  relationship  between 
nonequilibrium  steady  states  and  stochastic  pumps  have 
recently  been  addressed  by  two  other  groups.  Esposito  and 
Parrondo  [28]  have  shown  that  hidden,  rapidly  driven, 
reversible  stochastic  pumps  can  generate  currents  that — to 
an  observer  who  is  unaware  of  the  pumps — appear  to 
violate  detailed  balance.  Uzdin,  Fevy,  and  Kosloff  [29] 
have  established  a  thermodynamic  equivalence  between 
three  types  of  quantum  heat  engines,  in  the  limit  of  weak 
thermalization  and  a  weak  driving  field. 

A  different  context  in  which  similar  ideas  were  recently 
suggested  is  topological  insulators  [30,31],  where  temporal 
periodic  driving  can  be  used  to  mimic  material  properties 
that  are  difficult  to  achieve  in  equilibrium.  In  these  systems, 
commonly  denoted  as  “Floquet  topological  insulators,”  the 
properties  of  interest  are  not  the  probabilities,  currents,  and 
entropy  production  rates  as  in  our  case,  but  rather  the 
topological  classification  of  the  effective  Hamiltonian  and 
the  existence  of  protected  edge  states.  In  recent  years, 
Floquet  topological  insulators  have  been  experimentally 
demonstrated  in  various  setups  [32,33]. 

In  this  manuscript,  we  begin  by  considering  a  generic 
Markov  process  of  random  transitions  among  n  states  of  a 
system.  The  transition  rates  are  fixed  in  time  and  do  not 
satisfy  detailed  balance;  hence,  the  dynamics  lead  to  a 
NESS  with  nonvanishing  currents  and  entropy  production. 
We  then  show  how  to  prescribe  a  stochastic  pump  that  has, 
in  the  limit  of  many  cycles,  the  same  time-averaged 
probabilities,  currents,  and  entropy  production  rates  as 
the  NESS.  This  prescription  is  constructive  though  not 
unique.  Surprisingly,  the  construction  does  not  require  the 
solution  of  any  differential  equations,  only  linear  algebraic 
equations.  By  contrast,  a  mapping  in  the  opposite  direction 
(from  SP  to  NESS)  requires  that  we  first  determine  the 
periodic  state  of  the  system,  which  involves  solving  a  set  of 
coupled  ordinary  differential  equations  with  time-periodic 
parameters.  Typically,  this  can  only  be  done  numerically. 

In  some  applications,  transition  rates  are  constrained  by 
experimental  or  fundamental  considerations.  An  example 
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for  fundamentally  constrained  rates  is  kinetic  proofreading 
[34,35],  where  the  ratio  between  the  steady-state  proba¬ 
bilities  of  two  specific  states  is  controlled  without  manipu¬ 
lating  their  energies.  Kinetic  proofreading  provides  a  model 
for  understanding  how  low  error  rates  in  biomolecular 
synthesis  can  be  achieved  at  the  cost  of  hydrolyzing  ATP 
into  ADP  +  Pj.  As  an  application  for  the  tools  developed  in 
this  manuscript,  it  is  shown  that  kinetic  proofreading  in  a 
three-state  system  can  be  achieved  by  SP,  with  exactly  the 
same  controllability  over  the  time-averaged  probability 
distribution  and  at  exactly  the  same  cost  in  terms  of  entropy 
production  rates,  as  in  the  NESS  kinetic  proofreading. 

The  rest  of  the  manuscript  is  organized  as  follows:  In 
Sec.  IV,  we  formulate  the  problem  and  review  some  useful 
known  results.  We  then  consider  in  Sec.  V  the  simpler 
direction,  namely,  mapping  a  stochastic  pump  into  a  NESS. 
In  Sec.  VI,  we  present  two  types  of  transformations 
for  detailed  balance  matrices,  which  play  a  key  role 
in  our  construction  of  a  SP  that  mimics  NESS,  and  the 
“no-current-loops”  property  that  sets  a  constraint  on  pump¬ 
ing  with  a  time-dependent  detailed  balance  matrix.  The 
construction  of  a  pumping  protocol  is  described  in  Sec.  VII. 
As  an  application,  NESS  and  SP  kinetic  proofreading  of  a 
three-state  system  are  analyzed  in  Sec.  VI.  We  finish  with  a 
few  concluding  remarks  in  Sec.  VII. 

II.  SETUP  AND  BACKGROUND 
A.  Definitions 

We  consider  an  ergodic,  continuous-time  Markovian 
system  with  n  states.  The  evolution  of  the  system  consists 
of  random,  Poissonian  transitions  among  these  states, 
with  transition  rates  that  are  governed  either  by  a  time- 
independent  rate  matrix  1Z  (when  analyzing  nonequilibrium 
steady  states)  or  a  time-periodic  rate  matrix  W(f)  (for 
stochastic  pumps),  as  described  in  more  detail  below.  It  is 
convenient  to  picture  the  system  in  terms  of  a  “connectivity 
graph”  with  n  nodes  and  a  finite  number  of  edges  connecting 


FIG.  1.  A  four-state  system  described  by  a  graph.  Each  node 
represents  a  state  of  the  system.  The  edges  represent  nonvanish¬ 
ing  transition  rates  between  states.  In  this  example,  direct 
transitions  between  states  1  and  3  are  not  allowed. 


given  pairs  of  nodes.  An  edge  between  states  i  and  j  implies 
that  the  system  can  make  transitions  between  these  states 
(see  Fig.  1).  We  use  p(t)  to  denote  the  vector  of  the 
probabilities  whose  ith  component  /;,(r)  is  the  probability 
for  the  system  to  be  in  state  i  at  time  t. 

In  the  NESS  scenario,  the  evolution  of  the  system  obeys 
the  master  equation 


dtp  =  Up.  (1) 

For  i  j ,  the  matrix  element  77,;  is  the  probability  per  unit 

time  for  a  system  in  state  j  to  make  a  transition  to  state 
The  diagonal  elements  of  TZ  are  negative  and  are  deter¬ 
mined  by  conservation  of  probability:  J A7£,-;-  =  0.  For  the 
system  to  be  ergodic,  we  demand  that  (i)  if  7W  ±  0.  then 
also  TZjj  ±  0,  and  (ii)  the  graph  associated  with  TZ  is 
connected.  In  other  words,  for  any  pair  of  nodes  (states) 
j  and  i,  there  exists  a  path  from  j  to  i,  possibly  through  a 
sequence  of  intermediate  nodes,  along  the  edges  of  the 
graph.  Under  these  conditions,  Eq.  (1)  has  a  unique  steady- 
state  solution,  which  we  denote  by  pss,  and  any  solution  of 
Eq.  (1)  converges  to  this  steady  state  in  the  long-time 
limit  [36]. 

In  addition  to  the  steady-state  probabilities  pss,  we  are 
interested  in  the  steady-state  currents,  defined  by 

J))  -  TZijP?  -  lZJiPr,  (2) 

and  the  entropy  production  rates  associated  with  these 
currents  [36,37]: 


°sii  =  Ju  log 


KuP? 
KiiPr ' 


(3) 


We  assume  that  some  of  these  currents,  and  therefore  the 
corresponding  entropy  production  rates,  are  nonvanishing. 
In  other  words,  we  assume  the  dynamics  violate  detailed 
balance;  hence,  pss  is  a  genuinely  nonequilibrium  steady 
state. 

In  the  stochastic  pump  scenario,  the  system  obeys  a 
master  equation  with  a  time-periodic  rate  matrix. 


dtp  =  W(t)p,  (4) 


where 


W(t)  =  W(t  +  T)  (5) 

for  some  finite  period  T.  If  we  momentarily  treat  t  in  Eq.  (4) 
as  a  parameter  of  the  rate  matrix  (rather  than  as  the  time 
variable),  then  for  any  fixed  value  of  this  parameter,  we 
assume  the  rate  matrix  W(f)  has  a  unique  stationary 
solution  k (f),  and  we  further  assume  that 

Wij{t)itj{t)  -  =  0  V  i,j,  t.  (6) 

In  other  words,  the  dynamics  generated  by  W(t)  (for  fixed 
t)  satisfy  detailed  balance.  We  refer  to  nit)  as  the 
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equilibrium  state  of  W(r).  This  is  the  state  to  which  the 
system  would  relax  if  all  the  rates  W,;  were  “frozen”  in 
time.  Note  that  in  our  notation,  TZjj  denotes  a  nondetailed- 
balance  rate  matrix,  and  TV,-,-  represent  a  detailed-balance 
rate  matrix.  Therefore.  NESS  systems  are  associated  with  a 
rate  matrix  'TZlp  whereas  SP  systems  are  described  through 
a  periodic  W,y(f),  though  in  Sec.  VI,  we  also  refer  to  a 
time-independent  detailed-balance  rate  matrix  as  TV,,-. 

Let  us  now  return  to  thinking  of  t  as  time.  For  any 


(4),  the  quantities 

Jij(t)  =  Wijpj  -  wJiPi, 

(7) 

Oi 

«■**. 

II 

£ 

"23  "53 

(8) 

(suppressing  the  argument  t  on  the  right  side)  represent 
instantaneous  currents  and  entropy  production  rates.  Under 
Eqs.  (4)  and  (5).  the  statistical  state  of  the  system  evolves 
asymptotically  to  a  unique  time-periodic  state, 

pps(t)  =  pPs(t  +  T),  (9) 


quantities,  after  averaging  over  time: 

pT  p)s-  J'u  -  Jij-  <  <$■  (13) 

We  denote  the  above  problem  as  the  “forward”  problem. 
We  are  also  interested  in  the  “inverse”  problem:  Given  a 
time-dependent  detailed  balance  rate  matrix  W(f)  corre¬ 
sponding  to  the  time-averaged  quantities  pfs,  J  ,  and  off, 
we  want  to  construct  a  time-independent  rate  matrix  1Z  such 
that  Eq.  (13)  holds.  As  we  discuss  in  more  detail  below,  the 
solution  of  the  inverse  problem  follows  directly  from 
known  results;  therefore,  we  focus  mainly  on  the  forward 
problem  in  this  paper. 

When  7Z  and  W(f)  give  rise  to  dynamics  that  satisfy 
Eq.  (13),  we  say  that  the  stochastic  pump  “mimics”  the 
nonequilibrium  steady  state,  and  vice  versa. 

B.  Two  useful  decompositions 

The  two  well-known  decompositions  described  below — 
the  first,  an  algebraic  decomposition  of  rate  matrices,  and 
the  second,  a  topological  decomposition  of  the  connectivity 
graph — will  be  extensively  used  in  what  follows. 


with  currents  and  entropy  production  rates 


Jffit)  =  WijP?  ■ 
=  log 


-  WjiP?, 


WijPPj 


ps 


WjiP\ 


ps  • 


(10) 

(11) 


These  are  analogous  to  the  quantities  appearing  in  Eqs.  (2) 
and  (3),  only  ZffJ(t)  and  o'fj ( t)  are  periodic  with  time, 
whereas  Jf-  and  of-  do  not  vary  with  time. 

Throughout  this  paper,  we  are  interested  in  the  asymp¬ 
totic  properties  of  the  system;  therefore,  we  consider  only 
the  steady  state  pss  and  the  periodic  state  pps(t),  and  not 
the  process  of  relaxation  to  either  of  these  states. 

In  order  to  compare  the  NESS  and  stochastic  pump  (SP) 
scenarios,  let  us  define  the  time-averaged  quantities  in  the 
periodic  state  of  the  SP: 

W  =  jfTprw. 

=  t)dt. 

(12) 


The  problem  that  we  wish  to  study  can  now  be  formulated 
as  follows. 

Problem  formulation, — Given  a  time -independent  rate 
matrix  7Z  corresponding  to  steady-state  quantities  pss,  Jss, 
and  oss,  construct  a  time-periodic  detailed-balance  rate 
matrix  W(f)  whose  periodic  state  is  described  by  the  same 


1.  Rate  matrix  decomposition 

The  following  (unique)  decomposition  of  any  rate  matrix 
7Z,  obtained  by  Zia  and  Schmittmann  [38],  will  prove  to  be 
useful: 

n=(s+l-jss\v-1.  (14) 

Here,  the  multiplication  is  an  ordinary  matrix  multiplica¬ 
tion,  and  S  is  a  symmetric  matrix  whose  elements  in  each 
column  add  up  to  zero,  with  negative  entries  only  on  the 
diagonal.  Jss  is  the  antisymmetric  current  matrix  defined  in 
Eq.  (2),  and  V  —  diagfp")  is  a  diagonal  matrix  with 
elements  VH  =  pf  . 

An  immediate  corollary  of  Eq.  (14)  is  the  following 
statement:  If  a  rate  matrix  7 Z  is  the  product  of  a  symmetric 
rate  matrix  S  and  the  inverse  of  a  diagonal  matrix  V  (with 
positive  diagonal  entries  summing  to  unity),  then  7Z 
satisfies  detailed  balance;  i.e.,  there  are  no  currents  in 
the  stationary  state. 

2.  Cycle  decomposition 

The  currents  that  characterize  a  NESS  are,  in  general,  not 
independent  of  one  another,  as  they  must  satisfy  constraints 
arising  from  the  conservation  of  probability.  These  con¬ 
straints  embody  Kirchoff’s  law  of  currents.  The  cycle 
decomposition  method  provides  a  convenient  tool  to 
account  for  these  constraints  [36].  Briefly,  in  a  connected 
network  with  N  nodes  and  E  edges,  the  conservation  of 
probability  imposes  N  —  1  constraints  among  the  E  cur¬ 
rents  (one  current  per  edge).  It  is  then  convenient  to  identify 
C  =  E  —  N  +  1  fundamental  currents,  using  the  following 
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Pss=0.1 


Pss=0.4 


FIG.  2.  A  concrete  example  of  a  NESS  for  which  we  build  an 
equivalent  stochastic  pump.  In  this  example,  7 Ztj  ±  0  for  all  i,  j. 
The  spanning  tree  was  chosen  to  be  the  2-1,  2-3,  and  2-4  edges 
(dashed  red  lines),  and  the  fundamental  currents  are  the  currents 
along  the  1-4,  1-3,  and  3-4  edges  (solid  black  lines).  In  this 
system,  there  are  two  current  loops:  1  -»  3  -*•  4  -*•  1  and 
2  3  ->  4  ->  2. 


procedure.  First,  we  build  a  connectivity  graph  for  the 
system,  as  described  above  and  illustrated  in  Fig.  2  for  the 
case  of  four  nodes  and  six  edges  (hence,  C  —  3).  We  then 
construct  a  maximal  spanning  tree,  by  removing  C  edges 
without  breaking  the  connectivity  of  the  graph;  this  tree, 
illustrated  by  the  red  dash-dotted  lines  in  Fig.  2,  has  no 
cycles.  In  the  context  of  the  original  graph,  the  C  edges  that 
are  removed  to  form  the  spanning  tree  are  called  funda¬ 
mental  edges.  The  currents  along  these  edges  are  the 
fundamental  currents  (the  black  arrows  in  the  figure), 
and  the  currents  along  the  edges  of  the  tree  are  the 
spanning-tree  currents  (the  red  arrows  in  the  figure). 

For  NESS,  the  steady-state  currents  along  the  C  funda¬ 
mental  edges  can  take  on  any  values,  independently  of  one 
another.  However,  once  these  fundamental  currents  are  set, 
the  spanning-tree  currents  are  then  uniquely  determined  by 
conservation  of  probability:  The  sum  of  incoming  and 
outgoing  currents  at  each  node  must  vanish  in  the  steady 
state.  Therefore,  the  number  of  degrees  of  freedom  in  the 
matrix  Jss  is  not  (n2  —  n)/ 2  as  for  an  arbitrary  antisym¬ 
metric  matrix  but  is  determined  by  the  graph  topology. 

Unlike  NESS,  for  stochastic  pumps  the  fundamental 
currents  at  each  moment  do  not  fix  the  currents  on  the 
spanning-tree  edges  since  probabilities  can  temporarily 
accumulate  on  the  vertices  (Kirchoff’s  current  law  does 
not  apply  at  every  instant  of  time).  However,  Jps  on  the 
fundamental  edges  does  dictate  Jps  on  the  spanning  tree 
since  the  average  probabilities  are  conserved. 

III.  MAPPING  SP  TO  NESS 

In  this  section,  we  consider  the  inverse  problem  defined 
at  the  end  of  Sec.  II  A,  which  conceptually  is  the  simpler 
direction:  Given  a  time-dependent  periodic  rate  matrix 
W(f),  how  do  we  construct  a  time-independent  rate  matrix 
7Z  whose  steady-state  properties  satisfy  Eq.  (13)? 


To  construct  TZ,  we  first  solve  for  the  periodic  averages 
pfs ,  J'-- ,  and  afj  associated  with  W(f).  These  can  be 
calculated  by  obtaining  the  periodic  solution  of  the  master 
equation,  pfs(f),  and  then  plugging  this  solution  into 
Eq.  (12).  In  most  cases,  finding  the  periodic  solution  must 
be  done  numerically. 

Once  f/ff ,  and  of/  are  known,  we  must  build  a  rate 

matrix  1Z  whose  steady-state  properties,  pss,  Jss,  and  <fs, 
satisfy  Eq.  (13).  A  nice  consequence  of  Eq.  (14)  is  that  if  all 
the  currents  along  the  graph  edges  are  nonzero,  then  the 
quantities  pss,  Jss,  and  <x”  uniquely  determine  7 Z.  We 
express  this  relationship  by  the  shorthand  notation 

{pss,  Jss,  ass}  =►  n.  (15) 

To  see  this,  we  use  Eq.  (14)  to  write  the  entropy  production 
rates  as 


<  =  Jii 


C  _  l  rrss 
U(1  9  i  i 


(16) 


By  this  equation,  the  elements  of  Jss  and  oss  uniquely 
determine  S,  if  the  currents  are  nonzero: 


{J”,*"}  =>S.  (17) 

If  some  of  the  currents  are  zero,  then  the  corresponding 
elements  of  S  are  not  uniquely  determined  from  the 
currents  and  entropy  production  alone,  and  they  can  be 
arbitrarily  chosen.  Once  S  has  been  obtained  consistently 
with  Jss  and  ass,  it  can  be  combined  with  V  and  Jss,  via 
Eq.  (14),  to  give  7 Z. 

In  the  remainder  of  this  paper,  we  address  the  forward 
problem,  namely,  how  to  construct,  for  a  given  NESS,  a 
mimicking  SP  protocol.  While  this  problem  is  conceptually 
more  complicated  than  the  inverse  problem  discussed 
above,  it  turns  out  that  it  is  computationally  simpler  and 
does  not  require  any  solution  of  differential  equations. 

IV.  KEY  IDEAS 

Here,  we  establish  three  technical  results  that  will  play  a 
crucial  role  in  the  construction  of  W(f). 

A.  No  current  loops  in  detailed  balance  systems 

A  system  satisfying  detailed  balance  has  nonvanishing 
currents  when  the  instantaneous  probability  distribution 
differs  from  the  equilibrium  state  of  the  instantaneous  rate 
matrix.  These  currents,  however,  cannot  form  a  current 
loop.  In  other  words,  no  loop  i,j,  k, ....  in,  i  on  the  graph 
associated  with  the  system  can  have  all  the  currents 
oriented  in  the  same  direction  around  the  loop.  As  an 
example  of  a  current  loop,  consider  the  system  described  in 
Fig.  2.  No  detailed  balance  system  can  have  instantaneous 
currents  equal  to  the  currents  in  the  loop  1  — 3  — 4  — >  1 
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(or  in  the  loop  2  -»■  3  -►  4  ->■  2)  since  they  all  have  the 
same  orientation. 

To  see  why  a  current  loop  is  inconsistent  with  detailed 
balance,  let  us  consider  the  currents  along  the  edges  of  a 
loop  i,j,k . m,  i.  Using  Eq.  (14)  to  decompose  the 


detailed  balance  rate  matrix  W  =  <SII  1 
generated  by  a  distribution  q  satisfy 

,  the  currents 

Jij  —\  —\ 

— -  =  jij  qj  —  ni  qj, 

<-> U 

(18) 

J jk  -1  -1 

=  xk  qk-Kj  qj, 

6 jk 

(19) 

J  mi  —\  —\ 

V  -  nmlqm • 

^  mi 

(20) 

Summing  these  equations,  we  get  that  /S)  =  0 
around  the  loop.  This  means  that  not  all  the  currents  can 
have  the  same  sign  since  each  <S,;-  >  0.  Therefore,  there  are 
no  current  loops.  (A  similar  argument  was  used 
in  Ref.  [20]). 

By  contrast,  the  steady-state  currents  of  a  NESS  must 
form  at  least  one  current  loop.  To  see  this,  just  choose  a  site 
(denoted  by  i)  through  which  some  of  the  steady-state 
currents  flow.  Conservation  of  probability  implies  that  at 
least  one  of  these  currents  is  going  out  of  site  i,  so  we 
choose  such  a  current,  say,  from  i  into  site  j.  Now,  from  site 
j  again,  there  is  at  least  one  current  going  out,  say,  to  site  k. 
Following  the  same  argument,  from  any  site,  we  can  “flow” 
with  a  current  into  a  new  site,  but  since  the  number  of  sites 
is  finite,  after  no  more  than  n  such  steps,  we  must  come 
back  to  a  site  we  already  visited.  Therefore,  there  must  exist 
at  least  one  current  loop. 

B.  Transformation  for  the  diagonal  part 
of  the  decomposition  of  W 

Suppose  we  have  a  detailed  balance  rate  matrix  VV  and 
two  instantaneous  probability  distributions  q  and  p,  neither 
of  which  necessarily  corresponds  to  a  stationary  distribu¬ 
tion.  We  would  like  to  transform  VV  into  a  different  detailed 
balance  rate  matrix,  W,  such  that  VV  q  =  Wp.  This  is 
achieved  by  the  following  transformation: 

W  =  WQV~\  (21a) 

where  V  and  Q  are  the  diagonal  matrices  corresponding  to 
p  and  q,  respectively.  Using  indices,  this  reads 

W,v-  —  VVijqjPj  1 .  (21b) 

The  transformation  given  by  Eq.  (21a)  affects  only  the 
diagonal  part  of  the  decomposition  and  has  the  following 
properties: 


(1)  If  VV  satisfies  the  detailed  balance  condition, 
then  so  does  W.  This  can  be  seen  by  using  the 
decomposition  of  VV  as  in  Eq.  (14)  in  the  above 
transformation  and  noting  that  W  is  a  symmetric  rate 
matrix  times  a  diagonal  matrix;  therefore,  it  has  no 
currents  in  its  steady  state. 

(2)  The  instantaneous  currents  of  a  system  described  by 
VV  with  probabilities  q  are  the  same  as  those  of  a 
system  described  by  W  with  the  probabilities  p. 
This  follows  by  substituting  Eq.  (21b)  into  Eq.  (7). 

(3)  From  Eqs.  (8)  and  (21b),  it  follows  that  the  instanta¬ 
neous  entropy  production  rates  along  each  edge  (c^) 
for  a  system  described  by  VV  with  probabilities  q  and 
for  a  system  described  by  W  with  probabilities  p  are 
the  same. 

The  significance  of  this  transformation  can  be  stated  as 
follows.  If  we  have  a  rate  matrix  VV  and  probabilities  q, 
which  produce  instantaneous  currents  J  and  entropy 
production  rates  a ,  then  for  any  other  probability  distribu¬ 
tion  p,  we  can  construct  the  rate  matrix  W  that  generates 
the  same  J  and  a. 

C.  Transformation  for  the  symmetric  part 
of  the  decomposition  of  W 

Currents  arise  in  a  system  described  by  a  detailed 
balance  W  when  the  instantaneous  probability  distribution 
p  differs  from  the  equilibrium  distribution  n.  We  next  show 
how  to  vary  the  magnitudes  of  these  currents  (but  not  their 
directions)  while  keeping  p  and  h  fixed.  As  the  directions 
of  the  currents  do  not  vary  under  this  transformation,  no 
loops  can  be  formed  in  accordance  with  the  “No  loop 
condition”  in  Sec.  IV A. 

Let  us  use  the  decomposition  W  =  6TU1 ,  where  S  is 
symmetric  and  II  =  diag(ir).  The  currents  are  then  given  by 

Jij  =  Sij{n-lPj  -  nj'pi).  (22) 

We  see  that  by  varying  5,,-,  we  vary  the  magnitude  of  the 
current  but  not  its  sign,  since  <S,;  >  0. 

This  transformation  is  complementary  to  the  one  given 
by  Eq.  (21):  It  enables  us  to  adjust  the  currents  and  entropy 
production  rates  while  keeping  the  probabilities  p  and  Ji 
fixed,  by  tuning  the  symmetric  part  of  W  in  the  decom¬ 
position  given  by  Eq.  (14).  By  contrast,  with  the  previous 
transformation,  we  can  vary  the  probabilities  p  and  n  at 
fixed  currents  and  entropy  production  rates,  by  tuning  the 
diagonal  part  of  W. 

V.  CONSTRUCTION  OF  THE 
PUMPING  PROTOCOL 

Given  a  rate  matrix  1Z,  one  can  calculate  its  steady  state 
pss  (the  null  eigenvector  of  1Z)  and  thus  Jss  and  oss  using 
Eqs.  (2)  and  (3).  Our  goal  is  to  construct  a  periodic 
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pumping  protocol — a  time-dependent  detailed  balance  rate 
matrix  W(f)  with  some  period  T — such  that  Eq.  (13)  holds, 
in  other  words,  a  SP  that  mimics  the  NESS.  For  simplicity, 
we  assume  that  there  are  no  edges  along  which  the  steady- 
state  currents  are  zero.  The  case  involving  zero  currents 
along  some  edges  is  analyzed  in  the  Appendix  B. 

We  note  that,  in  general,  there  are  many  SPs  that  mimic 
any  specific  NESS — the  mapping  is  not  one  to  one.  Out  of 
the  many  SPs  that  mimic  the  NESS,  we  choose  one  using  a 
relatively  simple  construction,  yet  generic  enough  to  mimic 
any  NESS.  Naively,  we  would  like  to  have  a  SP  whose 
periodic  state  gives  rise  to  time-independent  quantities, 
Pf’(t)=p ?,  and  o?f(t)=o%.  Such  a 

construction  is,  unfortunately,  impossible.  This  can  be  seen 
from  the  result  of  Sec.  IV A,  which  states  that  for  all  t, 
J'-j(t)  cannot  have  any  current  loops,  whereas  Jf-  must 
have  at  least  one  current  loop.  Thus,  we  can  only  hope  to 
achieve  a  mapping  between  the  time-averaged  quantities 
associated  with  the  SP  and  those  of  the  NESS,  as  in 
Eq.  (13).  This  also  implies  that  at  least  some  of  the  currents 
of  the  SP  must  be  time  dependent. 

Though  it  was  the  simplest  we  could  find,  the  con¬ 
struction  described  below  is  nevertheless  somewhat  con¬ 
voluted.  We  therefore  first  give  an  overview  before 
proceeding  to  the  detailed  description.  The  main  reason 
for  the  complication  is  the  fact  that  the  periodic  solution 
pfs(t)  is  a  highly  nontrivial  function  of  the  pumping 
protocol  W(t).  To  avoid  this  complication,  we  simulta¬ 
neously  construct  both  the  pumping  protocol  W(f)  and  its 
periodic  solution  This  is  achieved  in  five  steps. 

In  the  first  step,  described  in  Sec.  VA,  we  divide  the 
pumping  protocol  temporal  interval  of  duration  T  into  two 
equal  half-intervals,  designated  as  a  and  b.  We  then  assign 
an  arbitrary  fixed  detailed-balance  matrix  W“  and  an 
arbitrary  fixed  probability  distribution  qa,  to  be  associated 
with  the  first  half-period  of  driving.  We  refer  to  VVa  and  qa 
together  as  the  seed  for  that  half-cycle,  and  these  quan¬ 
tities  will  be  used  to  set  the  current  directions  during  that 
time  interval.  Next,  the  seed  and  current  directions  for  the 
first  half-cycle  are  used  to  assign  a  seed  (Wb,  qb)  and 
current  directions  for  the  second  half-cycle.  The  current 
directions  during  the  first  half-cycle  are  opposite  of  those 
of  the  second  half-cycle,  and  both  sets  are,  by  construc¬ 
tion,  consistent  with  the  no-current-loop  condition  in 
Sec.  IV  A. 

In  the  next  two  steps,  we  use  the  seeds  to  construct  fixed 
sets  of  currents  (J Y  and  J7”f  )  and  entropy  production  rates 
(of-  and  of)  for  the  first  and  second  halves  of  the  cycle, 
whose  averages  over  the  two  halves  are  equal  to  the  steady- 
state  values  that  we  wish  to  mimic: 

+  +  4)  (23) 


for  all  i  ±  j.  This  is  done  first  for  the  fundamental  currents 
in  Sec.  VB  and  then  for  the  spanning-tree  currents  in 
Sec.  VC. 

Up  to  this  point,  the  currents  and  entropy  production 
rates  for  the  two  half-cycles  have  been  constructed  from  the 
initial,  arbitrary  seeds,  but  the  corresponding  time-periodic 
rate  matrices  Wa(f)  and  Vvh ( t)  that  actually  generate  these 
currents  and  entropies  are  not  yet  known.  In  the  fourth  step, 
described  in  Sec.  V  D,  we  use  the  transformation  of  Sec.  IV 
C  to  adjust  the  symmetric  part  of  the  seed  matrices,  Wa'b, 
arriving  at  new,  time-independent  rate  matrices  \Va,h  that 
produce  the  desired  currents  Ja,b,  for  the  seed  probabilities 
qa-b .  These  currents,  together  with  the  desired  averaged 
probabilities  p fs,  fix  p?s(t).  In  the  last  step  (Sec.  V  E),  we 
use  the  transformation  of  Sec.  IV  B,  together  with  the 
symmetric  parts  of  Wa'b ,  to  construct  time-dependent  rate 
matrices  W" ( t)  and  TV* ( t)  for  which  pfs(t)  is  the  periodic 
solution  of  the  master  equation. 

In  the  specific  protocol  described  below,  the  entries  of 
the  matrix  W(f)  are  not  continuous  functions  of  time,  as 
they  have  discontinuities  between  the  two  T/ 1  intervals. 
These  discontinuities  are  not  essential  and  can  be  removed 
at  the  expense  of  making  the  construction  less  transparent. 

For  clarity  of  presentation,  some  of  the  formal  definitions 
of  the  construction  are  followed  by  a  concrete  application  to 
the  example  of  the  four-state  system  described  in  Fig.  2.  In 
this  example,  the  NESS  system  has  four  states  with 
pss  =  (0.1, 0.2, 0.3, 0.4).  The  fundamental  currents  were 
chosen  to  be  ^7'”,  =  3,  =  2,  and  =  1.  The  currents 

for  the  spanning-tree  edges  are  then  dictated  by  Kirchoff’s 
law — the  sum  of  currents  in  each  vertex  must  be  zero.  The 
corresponding  current  matrix  is 


J 


SS  _ 


1  0 
-2 
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-1 


-3 
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1  \ 

1 

-2 

°J 


(24) 


Finally,  we  choose  the  entropy  production  rate  to  be  1  along 
all  the  edges:  ol;  =  1  for  all  i  ±  j.  Using  Eq.  (16),  the 
matrix  S  can  be  calculated,  and  it  is  given  in  Eq.  (A2)  in 
Appendix  A.  The  matrix  7 Z  giving  rise  to  this  particular 
NESS  can  be  constructed  using  Eq.  (14). 


A.  Step  1:  Choosing  the  seed 

In  what  follows,  superscripts  a  and  b  indicate  quantities 
associated  with  the  first  and  second  halves  of  the  period, 
respectively. 

Our  first  step  is  to  choose  an  arbitrary  detailed  balance 
matrix  on  the  graph.  This  is  done  by  choosing  an  equilib¬ 
rium  state  for  the  first  half-period  na  and  a  symmetric  rate 
matrix  S  from  which  we  compose  W“  =  cS(na)_1 ,  where 
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nfl  =  diag (na).  Next,  we  choose  a  fixed  probability  dis¬ 
tribution  qa  ±  7ca  that  satisfies 


log 


qaiJt“ 


< 


log 


KijP? 


(25) 


for  any  i  f  j.  It  is  always  possible  to  satisfy  this  condition 
by  choosing  qa  close  enough  to  n" .  As  we  see  in  the  next 
section,  this  condition  is  necessary  for  the  consistency  of 
our  construction. 

For  the  second  half  of  the  period,  we  replace  h“  and  qa 
by  vectors  with  components  if  =  l/n" — from  which  we 
construct  W1'  =  >S(nfo)_1 — and  qb  =  1  /q°.  The  two  prob¬ 
ability  vectors  nb  and  qh  are  not  normalized,  but  as  will 
become  clear,  this  normalization  does  not  play  any  role;  it 
will  prove  to  be  simpler  to  work  with  these  unnormalized 
vectors.  We  note  that  the  currents  generated  by  W“  and  qa. 


=  (26) 

have,  on  each  edge,  opposite  signs  to  those  generated  by 
Wb  and  qb,  given  by 


Jij  =  stj 


*i<ti 


(27) 


The  values  of  J'lj’  will  not  explicitly  be  used  in  what 
follows — only  their  directions,  which  by  construction  are 
consistent  with  detail  balance.  We  additionally  note  that 


B.  Step  2:  Fundamental  edge  currents 

In  this  step,  we  set  the  currents  along  the  fundamental 
edges  to  be  constant  during  each  of  the  two  half  cycles, 
such  that  their  time  averages  (the  average  between  the  first 
and  second  halves)  is  equal  to  Jss ,  and  the  time  average  of 
the  entropy  production  rates  is  equal  to  ass .  Moreover,  we 
choose  these  fundamental  currents  to  have  the  same 
directions  as  the  fundamental  currents  in  Ja  during  the 
first  half-cycle  and  to  be  reversed  in  direction  during  the 
second  half-cycle. 

We  first  make  sure  that  for  each  fundamental  edge,  the 
average  of  3“-  and  3\j,  and  therefore  the  time-averaged 
current,  is  exactly  J\s-.  To  that  effect,  we  introduce  the 
following  rules: 

(i)  If  the  direction  of  the  NESS  current,  Jf-,  is  the  same 

as  the  direction  of  3u  defined  above,  then  we  set  in 
the  first  half  of  the  period  J /  =  (2  +  0^)3™  and  in 
the  second  half  of  the  period  3bn  =  a^fT)].  where 

oijj  are  positive  and  will  be  determined  below. 

(ii)  If  the  direction  of  the  NESS  current,  3sf,  is  not  the 

same  as  the  direction  of  3f  then  we  set  in  the  first 
half  of  the  period  and  in  the  second 

half  J$j  =  (2  +  aij)J%. 

According  to  this  rule,  the  directions  of  the  currents  during 
the  first  half  of  the  period  are  the  same  as  those  of  J’lj  and 
are  opposite  to  those  in  the  second  half.  Moreover,  by  the 
above  construction,  the  time-averaged  currents  on  the 
fundamental  edges  have  the  required  values 


—a  „ci  —b  „b 

,  *i  qj  ,  71  i  qj 

log77T7  =  -log 


clini 


(28) 


-y“]  +  3bj)  =  3rj. 


(30) 


Normalizing  the  probabilities  nb  and  qb  does  not  change 
Eq.  (28),  nor  the  sign  of  J f,-  in  Eq.  (27).  Thus,  working 
with  the  non-normalized  probabilities  is  justified. 

To  illustrate  this  part  of  the  construction  with  our  four- 
state  example,  we  choose 


/  — 3  1  1  1  \ 

(  1/4  \ 

1-311 

- 

1/4 

11-31 

9  71  — 

1/4 

1  1  1  —3/ 

\  1/4/ 

We  emphasize  that  both  S  and  na  are  arbitrary. 

We  next  note  that  min,-  j  [log  TZijPf  —  log  7 Zjjpf]  =  1/3. 
Therefore,  if  we  choose  qa  such  that  |  log  nat  —  log  9f\< 
1/6,  then  Eq.  (25)  is  satisfied.  Any  vector  close  enough 
to  n“  will  do.  As  an  example,  we  use 
qa  —  (0.23, 0.24, 0.26, 0.27).  In  the  second  half  of  the 

period,  these  correspond  to  nb  =  (4, 4, 4, 4)  and  qb  = 
(4.3478, 4.1667,  3.8462, 3.7037). 


Next,  we  determine  the  values  of  the  a,-,  ’s  so  as  to  satisfy 
the  requirement  on  the  entropy  production  rates.  Assuming, 
for  the  moment,  that  the  probability  distributions  in  the  first 
and  second  halves  of  the  period  are  given  by  If  and  If  and 
that  the  equilibrium  distributions  of  the  detailed  balance 
matrices  during  the  first  and  second  halves  of  the  period  are 
given  by  na  and  nh ,  respectively,  then  the  entropy  pro¬ 
duction  rates  with  the  currents  J /■  and  3b-  during  the  two 
halves  of  the  cycle  are  given  by 


=  Jff0  g 


*j4j 

nc;q“  ' 


—b  „b 

^ b  _  Tb  r 71  i  qi 


=  J?i  log- 


'lJ  ~,]"Dnbq“ 


(31) 


Substituting  in  these  equations  J /  and  Jb-  in  terms  of 
and  demanding  that  \  (o'/  +  cf  )  =  o”,  we  get  an  equation 
for  (ijj  whose  solution  is 
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logoff1  l°g 


nuP) 


KupT 


- 1. 


(32) 


Equation  (25)  ensures  that  indeed  a,-  ■  >  0. 

To  illustrate  this  step,  we  examine  the  signs  of  the 
currents  generated  by  the  matrix  W“  =  <S(na)-1  and  the 
probability  qa,  denoted  by  J" ,  on  the  fundamental  edges: 

sign  (J13)  —  +1» 
sign  (^14)  =  +1, 

sign  (1/34)  =  +1.  (33) 

While  the  direction  of  the  current  along  the  1-4  edge  is  the 
same  as  the  current  orientation  of  Jss ,  for  the  other  two 
fundamental  edges,  the  directions  of  Ja  and  Jss  are  not 
the  same. 

We  next  solve  Eq.  (32)  for  a,-,-.  The  explicit  expression, 
as  well  as  the  currents,  is  given  in  the  Appendix  A.  Figure  3 
shows  the  currents  on  the  first  half-period  (left)  and  second 
half-period  (right).  Note  that  (i)  the  direction  of  the  currents 
along  each  edge  is  opposite  in  the  two  halves  of  the  period, 
and  (ii)  as  discussed  above,  in  the  first  half-period  the 
direction  of  the  current  along  the  1-4  edge  is  the  same  as 
that  of  Fig.  2,  but  the  direction  of  currents  along  the  other 
fundamental  edges  is  different  from  that  of  Fig.  2,  and 
(iii)  there  are  no  current  loops  in  Fig.  3. 


These  constraints  ensure  that  we  indeed  have  a 
periodic  time  evolution:  A/;,-  =  d,Pidt  = 
(T/2)J2j(J'ij  +  J ij )  —  0-  Note  that  these  are  only 
n  —  1 -independent  equations  since  conservation  of 
probability  adds  the  constraint  =  0. 

(2)  For  each  spanning-tree  edge,  we  demand  that 


Jr,  log 


Jbu  log 


=  2^. 


(35) 


The  number  of  edges  in  the  spanning  tree  is  n  —  1, 
and  therefore,  these  are  n  —  1  additional  equations. 
They  ensure  that  the  time-averaged  entropy  produc¬ 
tion  rate  along  the  i,  j  edge  is  the  same  as  a™. 
Equations  (34)  and  (35)  are  2 (n  —  1)  linear  equations  for 
2(n  —  1)  unknowns:  Ja'h  on  the  spanning  tree,  which  has 
n  —  1  edges  [39].  Moreover,  the  directions  J“J  that  solve 
these  equations  are  the  same  as  those  of  J"f,  respectively. 
To  see  this,  let  us  use  the  definition  of  qb  and  nb  in  the 
second  condition  above,  together  with  the  definition  of  tyfj 
[Eq.  (3)]: 


(Jf;  -Jf;)  =  2^  log 


r 

’di< 


log 


n 


KjiP? 


(36) 


Taking  the  absolute  value  of  both  sides  in  the  above 
equation  and  using  Eq.  (25)  implies  that 


C.  Step  3:  Spanning-tree  currents 

So  far,  we  have  described  how  to  construct  the  funda¬ 
mental  edge  currents.  Next,  we  discuss  the  currents  along 
the  edges  of  the  spanning  tree.  For  both  the  first  and  the 
second  half-periods,  we  impose  the  following  two 
constraints: 

(1)  The  sum  of  currents  feeding  into  any  site  i  during  the 
first  half-period  must  be  equal  to  minus  the  same 
quantity  during  the  next  half-period: 

E-A = -E  Ji-  (*>) 

j  j 


J'ij  J'h  >  2  J-'ij  ■  (37) 

However,  by  the  construction  of  J "■  and  j\-,  their  average 
is  equal  to  the  steady-state  current,  Jfr  therefore, 

+  =  (38) 

Equations  (37)  and  (38)  are  consistent  with  each  other  only 
if  the  sign  of  J '(  is  opposite  to  that  of  Jhir  Moreover,  as 
afj  >  0  [this  follows  from  Eq.  (16)],  the  signs  of  J "  and 
J1’-  must  be  the  same  as  those  of  log  n‘‘q(j  / q'-x'l  and 
log  n1- qb / q1- n'j ,  respectively;  otherwise,  the  left-hand  side 


FIG.  3.  The  currents  of  the  stochastic  pump  constructed  to  mimic  the  NESS  in  Fig. (2).  The  pumped  currents  during  the  first  (left)  and 
second  (right)  half-periods  are  shown.  On  each  of  the  edges,  the  current  directions  in  the  two  half  periods  are  opposite.  Note  that  there 
are  no  current  loops  in  both  of  the  half  periods. 
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of  Eq.  (35)  will  be  negative.  Therefore,  it  is  also  the  same  as 
the  sign  of  Jatj. 


D.  Step  4:  Transforming  the  symmetric  part 
for  the  currents 

At  this  stage,  we  have  constructed  the  time-dependent 
currents  for  both  half-cycles  with  the  same  directions  as  the 
currents  of  W"J‘  with  qa-b.  We  can  now  use  the  trans¬ 
formation  of  the  symmetric  part  of  the  decomposition  of  the 
rate  matrix  (defined  in  Sec.  IV  C)  to  adjust  the  symmetric 
part  of  W  such  that  the  currents  generated  by  the  trans¬ 
formed  matrix  and  the  seed  probabilities  qa-b  have  the 
constructed  values 


S?;  = 


7a. 
J  ‘J 


‘J  (<)  ‘<7“’ 


Jb. 

<rb  _  _ 

0  („b\-\„b 


,b\-l„b  ' 


q j~W)  Tt 


(39) 


Importantly,  all  these  off-diagonal  elements  are  positive. 
This  follows  from  the  fact  that  the  denominators  are  just 
/Si j,  but  by  our  construction,  the  sign  of  is  the 
same  as  that  of  J/f  ■  The  diagonal  elements  of  Sa,b  are  now 
determined  by  the  requirement  that  the  sum  of  each  of  the 
columns  is  zero. 


E.  Step  5:  Forming  a  solution  to  the  master  equation 

We  have  now  arrived  at  the  matrices  VV"'/'  = 
S“-b(na’b)-'.  These  are  the  transformed  seed  matrices, 
which  have  been  constructed  (by  tuning  the  elements  of 
S“-b)  so  as  to  produce  the  desired  currents  and  entropy 
production  rates  when  the  probability  vectors  are  qa-b. 
However,  the  time  average  of  qa  b  is  not  pss,  and  in  fact, 
qa’b  are  not  solutions  of  the  master  equation: 
dtqa,b  ri  W'J’q“'h.  To  remedy  this  situation,  we  use  the 
transformation  of  the  diagonal  part  of  the  decomposition  of 
TV,  defined  in  Sec.  IV  B. 

First,  we  want  the  time-averaged  probabilities  to  be  equal 
to  pss.  Second,  we  already  know  what  d,  p  ,■  should  be,  in 
terms  of  the  desired  currents  Jah .  Namely,  d,p,  =  Y/ )J ij ■ 
Therefore,  we  do  the  following: 

(1)  Calculate  ma-b  =  dtp  =  Wa'bqa'b.  These  are  the 
temporal  slopes  of  the  probabilities  that  solve  the 
master  equation,  during  the  first  and  second  halves 
of  the  period. 

(2)  Choose  7  such  that  0  <  p\s  ±  (T/4)m“'b  <  1  for 
any  i.  This  choice  ensures  that  the  probabilities  stay 
bounded  between  0  and  1 ,  and  is  always  possible  by 
taking  7  to  be  small  enough. 

(3)  Construct  p/{t)  =  pf  -  (7/4 )mf  +  m/t  and 
pb(t)  —  pf  —  (37/4)mb  +  mbt.  These  are  the 


solutions  of  the  master  equation  in  the  first  and 
second  halves  of  the  period,  respectively.  It  follows 
from  Eq.  (34)  that  mb  =  —m‘‘.  This  further  implies 
that  the  probabilities  defined  above  are  continuous 
functions  of  time,  i.e.,  p/(7/2)  =  p/(7/2)  and 
p/( 0)  =  pb{T),  as  they  should  be. 

(4)  From  the  above,  we  define 


S“(na)-lQa{va)~l{t) 

Sb(Ub)~1Qb(Vb)-1(t) 


t  <  7/2 
t  >  7/2. 


(40) 


The  matrix  W(f)  has  all  the  periodic  state  averages  we 
demand,  and  its  periodic  state  solutions  in  the  two  halves  of 
the  period  are,  by  construction,  pa-b(t). 

For  our  example,  we  calculate  the  slopes  dtp  on  the  two 
half-cycles  by  proper  summation  of  the  currents:  drpa  — 
(37.89,-2.51,5.85,-41.22)  and  <9^  =  (-37.89, 2.51, 
—5.85,41.22),  which,  as  expected,  are  equal  in  magnitude 
and  opposite  in  sign.  To  satisfy  0  <  Pi(t)  <  1,  we  must 
choose  7  small  enough,  say,  7  =  0.01.  Using  7,  we  can 
obtain  the  time-dependent  probability  distributions  pa-b(t). 
These  linear  functions  are  plotted  in  Fig.  4.  Plugging  these 
into  Eq.  (40)  gives  W(f).  To  verify  that  the  solution  of  the 
master  equation  with  the  constructed  W(f)  has  the  required 
properties,  we  solve  this  system  numerically.  The  numeri¬ 
cal  results  for  p(t),  shown  as  red  circles  in  Fig.  4,  agree  (up 
to  numerical  error)  with  the  analytical  solution. 

F.  Remarks 

We  conclude  this  section  with  a  few  comments  on  our 
construction.  First,  the  protocol  presented  above  is  clearly 
not  unique.  For  example,  different  seeds  or  choices  of  a 
spanning  tree  result  in  different  protocols.  The  nonunique¬ 
ness  might  be  used,  in  principle,  to  match  additional 
quantities,  e.g.,  fluctuations  around  the  average  or  the  rate 
of  decay  towards  the  steady  and  periodic  states.  In  addition, 


FIG.  4.  p(t)  of  the  proposed  construction  for  our  four-state 
example.  The  blue  line  is  the  exact  (constructed)  solution,  and  the 
red  dots  represent  the  numerical  solution  of  the  master  equation. 
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as  mentioned  previously,  in  our  construction,  W(t)  has 
discontinuities  between  the  two  halves  of  the  period.  This 
results  from  discontinuities  in  S,  n,  and  Q.  The  disconti¬ 
nuities  in  S  can  be  avoided  if  the  currents  are  not  taken  to 
be  fixed  during  the  two  halves  of  the  period  but  changing 
with  time,  such  that  the  currents  at  t  —  T/2  vanish.  The 
discontinuities  in  n  and  Q  can  be  avoided  if  we  change  q  as 
a  function  of  time,  crossing  n  at  t  =  T /l.  This,  however, 
makes  the  construction  more  cumbersome.  Next,  we  note 
that  in  the  above  construction,  both  the  symmetric  and  the 
diagonal  parts  of  the  decomposition  of  W  change  with 
time.  This  is  known  to  be  an  essential  feature  of  all 
pumping  protocols  [21]. 

Lastly,  we  note  that  for  NESS,  there  is  no  minimal 
entropy  production  rate  associated  with  a  given  set  of 
currents,  as  is  evident  from  Eq.  (16):  For  any  J,  the  entropy 
production  rates  of  the  NESS  can  be  made  arbitrarily  small 
by  taking  S  to  be  sufficiently  large  [38].  Our  mapping 
implies  this  is  also  the  case  for  stochastic  pumps:  Finite 
currents  can  be  pumped  with  arbitrarily  small  values  of 
dissipation.  This  is  surprising,  though  a  similar  result  was 
obtained  in  the  context  of  hidden,  reversible  pumps  [28]. 
Stated  more  generally,  for  any  connected  graph,  both  a 
NESS  and  a  SP  can  be  constructed  to  have  any  desired  set 
of  (time-averaged)  probabilities,  nonzero  currents,  and 
positive  entropy  production  rates,  provided  the  currents 
obey  Kirchhoff’s  law. 

Let  us  now  consider  the  construction  of  a  stochastic 
pump  with  small  entropy  production  rates  but  large 
currents.  For  the  entropy  production  rates  of  the  NESS 
to  be  very  small  with  nonvanishing  currents, 
log  (TZjjpSjS /TZjjp^)  must  be  very  small,  say,  of  order  e. 
Equation  (25)  then  implies  that  log  (rf q“ / q“)  ~  O(e)  as 
well.  Exponentiating  this  gives  {iff q“ / n" q“)  =  1  +  0(e); 
thus,  {naj)~lqc-  -  (n;‘l)~lq°  ~  O(e).  By  Eq.  (39),  this 
implies  5  ~  0(e_1 ).  Therefore,  in  stochastic  pumps — as 
in  NESS — small  entropy  production  rates  with  nonvanish¬ 
ing  currents  come  at  the  cost  of  large  values  in  the 
symmetric  part  of  the  rate  matrix.  But  when  the  elements 
of  S  are  large,  the  corresponding  transitions  occur  very 
rapidly.  Thus,  we  obtain  finite  currents  at  arbitrarily  low 
dissipation  when  there  is  a  large  separation  of  time  scales, 
with  transitions  among  the  n  states  of  the  system — and 
therefore  relaxation  to  equilibrium — occurring  much  more 
rapidly  than  the  external  driving  of  parameters.  The  greater 
the  separation  of  time  scales,  the  more  the  stochastic  pump 
approaches  the  adiabatic  (quasistatic)  limit  [5],  in  which  the 
system  remains  in  thermal  equilibrium  at  all  times  and  there 
is  no  entropy  production. 

VI.  APPLICATION:  KINETIC  PROOFREADING 

In  the  previous  section,  we  have  introduced  a  mapping 
from  any  NESS  to  a  SP  with  the  same  set  of  averaged 
currents,  probabilities,  and  entropy  production  rates. 


However,  this  construction  requires  the  ability  to  control 
all  the  elements  of  the  rate  matrix  W(f),  which  might  be 
experimentally  challenging,  or  even  impossible,  in  some 
applications.  Can  the  NESS-SP  mapping  be  achieved  even 
with  constrained  W(f)  ?  The  answer  to  this  question  clearly 
depends  on  the  details  of  the  constraints.  Nevertheless,  the 
tools  developed  so  far  might  be  useful  even  for  constrained 
systems.  We  next  demonstrate  this  through  a  concrete 
example:  kinetic  proofreading  in  a  simple  system.  In  this 
example,  the  constraints  are  set  not  by  experimental 
limitations  but  by  the  structure  of  the  problem  itself.  In 
the  next  subsections,  we  present  the  main  idea  of  kinetic 
proofreading  and  how  it  can  be  achieved  using  NESS.  We 
then  show  that  despite  the  constraints,  kinetic  proofreading 
has  a  NESS-SP  equivalence  in  terms  of  averaged  proba¬ 
bilities,  currents,  and  entropy  production  rates. 

A.  Kinetic  proofreading  by  NESS 
1.  Introduction  to  kinetic  proofreading 

Kinetic  proofreading  was  originally  introduced  by 
Hopfield  [34]  and  Ninio  [35]  to  explain  the  small  error 
rates  in  biochemical  reactions  such  as  protein  synthesis. 
The  system  in  kinetic  proofreading  is  described  by  a 
discrete-state  continuous-time  Markovian  model.  Each 
state  in  the  system  is  associated  with  an  energy  Eh  and 
the  transition  rates  between  the  states  are  of  Arrhenius  type, 

(  g-(sy_£j)  j  z/z  j 

=  \  -j:wik  i  =  j ,  (41) 

where  B,j  —  Bjj  is  the  height  of  the  barrier  between  the  z'th 
and  /tli  states;  for  simplicity,  we  work  in  units  where  the 
inverse  temperature  is  [)  —  1.  The  rate  matrix  W  defined 
above  is  detailed  balanced,  and  its  equilibrium  distribution 
is  given  by  the  Boltzmann  distribution, 

e~Ei 

n j  =  — — — ,  where  Z  =  e~Ei .  (42) 

i 

The  quantity  of  greatest  interest  in  kinetic  proofreading  is 
the  discrimination  ratio  D  defined  as  the  ratio  between  the 
steady-state  probability  to  be  in  one  of  the  states,  c  (the 
“correct”  state),  and  the  probability  to  be  in  some  other 
specific  state,  w  (the  “wrong”  or  incorrect  state).  The  reason 
why  D  is  the  important  quantity  (rather  than  the  probability 
to  be  in  the  correct  state)  is  that,  in  the  process  of  protein 
synthesis,  there  is  a  small  probability  per  unit  time  to  jump 
from  the  two  specific  states,  w  and  c,  into  the  next  stage  in 
the  synthesis  (which  is  not  part  of  the  system  described  so 
far).  This  probability  per  unit  time  is  assumed  to  be  small 
enough  not  to  have  any  significant  effect  on  the  evolution  of 
the  system  itself,  which  therefore  relaxes  into  its  steady 
state.  As  there  are  no  direct  transitions  to  the  next  stage 
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from  other  than  the  c  or  w  states,  the  error  rate  is  equal  to 
the  ratio  between  the  probabilities  to  be  in  states  c  and  w. 

If  the  system  is  in  equilibrium,  then  D  is  given  by 

£eq  =  -  =  ^-^.  (43) 

This  ratio  depends  only  on  the  energy  difference  between 
the  two  states.  Kinetic  proofreading  is  a  method  used  by 
nature  to  increase  the  discrimination  ratio  beyond  its 
equilibrium  value.  The  main  challenge  in  kinetic  proof¬ 
reading  is  therefore  to  control  the  discrimination  ratio 
without  changing  the  energies  Ec  and  Ew,  as  well  as  the 
barrier  between  them;  namely,  these  energies  and  banner 
are  given  as  constraints. 

For  simplicity,  in  what  follows,  we  consider  a  specific 
example  of  a  three-state  system,  shown  in  Fig.  5,  whose 
states  are  c,  w,  and  an  auxiliary  state  denoted  by  x.  This 
system,  which  is  the  minimal  system  in  which  detailed 
balance  can  be  violated,  is  not  the  standard  system  for 
kinetic  proofreading  in  biological  context,  which  has  five 
states  [40].  Nevertheless,  the  main  idea  can  be  demon¬ 
strated  through  this  example,  which  is  significantly  simpler. 

In  biological  systems,  kinetic  proofreading  is  achieved 
through  breaking  detailed  balance,  e.g.,  coupling  the 
w  4*  x  transition  to  the  hydrolysis  of  ATP  into  ADP, 
whose  concentrations  are  constantly  maintained  out  of 
equilibrium.  Such  a  coupling  modifies  the  Arrhenius  rates: 
Denoting  the  rate  matrix  now  by  7 Z  (as  it  is  not  detailed 
balanced  anymore),  its  off-diagonal  elements  are  the  same 
as  those  of  W,  i.e.,  77(/-  =  W,-,-,  except  for  the  wx  and  xw 
rates,  which  become 

nwx  =  W^l2,  (44) 

nxw  =  Wxwe-*'2.  (45) 


Ec  ■  Ew 


FIG.  5.  A  three-state  system,  through  which  we  demonstrate 
kinetic  proofreading.  The  system  has  three  states:  the  correct  state 
c,  the  wrong  state  w,  and  the  auxiliary  state  x.  Each  state  is 
associated  with  internal  energies  Ec,  Ew,  and  Ex,  respectively. 
Between  any  two  states,  there  is  a  (symmetric)  barrier, 
Bcw  =  Bwc,  Bcx  =  Bxc,  and  Bwx  =  The  transition  rates 
between  the  different  states  are  given  by  the  corresponding 
Arrhenius  rates. 


Here,  the  affinity  <p  is  the  Gibbs  free  energy  released  by  the 
hydrolysis  of  one  ATP  molecule,  and  it  can  be  viewed  as  a 
thermodynamic  force  that  drives  current  around  the  loop 
x  — ►  w  -*■  c  ->  x  [36].  In  principle,  this  affinity  could  have 
been  assigned  to  the  xc  edge,  or  split  between  the  xc  and 
xw  edges,  without  fundamentally  affecting  the  results.  As 
the  dynamics  no  longer  satisfy  detailed  balance,  the  steady- 
state  probabilities  pss  are  not  given  by  the  Boltzmann 
distribution;  thus,  kinetic  proofreading  is  possible. 


2.  Constructing  a  NESS  kinetic  proofreading  with 
arbitrary  steady-state  distribution 

Fet  us  demonstrate  that  given  the  energies  Ec,  Ew  and  the 
barrier  Bcw,  it  is  possible  to  achieve  any  desired  probability 
distribution  pss  as  the  steady  state  of  the  system,  by 
controlling  the  affinity  <p,  the  auxiliary  state  energy  Ex, 
and  the  barriers  between  the  auxiliary  state  and  the  other 
states — Bxc  and  Bxw.  In  other  words,  we  assume  that  the 
properties  of  the  two  states  c  and  w  and  the  transitions 
between  them  are  given,  and  our  goal  is  to  show  that  by 
choosing  the  affinity  and  the  auxiliary  state  properties,  it  is 
possible  to  achieve  any  probability  distribution  as  the 
steady  state  of  the  system.  To  show  this,  we  first  note  that 
there  is  only  a  single  current  in  the  system  since  there  is 
only  one  loop.  This  current  can  be  expressed  [substituting 
1Z  in  Eq.  (2)]  in  terms  of  the  known  quantities  Ec,  Ew,  Bcw 
and  the  desired  steady-state  probabilities  pscs  and  p  f: 

JSS  =  e-Bcw  ^gEc pss  _  eEw pssy  (46) 


Note  that  the  known  steady-state  probabilities  psf,  pscs,  the 
energies  Ec,  Ew,  and  the  barrier  Bcw  uniquely  define  the 
steady-state  current  Jss\  namely,  it  cannot  be  tuned  by 
changing  the  other  parameters.  In  the  steady  state,  the  same 
current  must  also  flow  through  the  cx  edge;  thus. 


Jcx  =  e  Bcx{eE*psx 


p  V 

-  e^psc 


=  JS 


(47) 


Given  pscs,  pxs,  and  Jss,  we  can  choose  Ex  such  that  the 
sign  of  eExpx  —  eE‘  /;”  is  the  same  as  the  sign  of  Jss ,  and 
once  Ex  is  chosen,  we  can  set  Bxc  to  be 

sjSS 

Bxc  =  —  In— = - - — = - . 

eEcpscs  ~  eE'psxs 

Next,  we  can  set  the  affinity  f  such  that  the  current  in  the 
xw  edge  is  in  the  same  direction  as  in  the  cw  edge:  Using 

J"x  =  e-^ie^-ttpf  -  g^+^p")  =  Jss,  (48) 

we  can  first  choose  f  such  that  the  sign  of  eEx+^  px  — 
eE«-ht‘ psf  agrees  with  the  sign  of  Jss,  and  then  calculate 
Bwx  such  that  the  above  equation  holds. 

So  far,  we  have  seen  that  by  choosing  the  energy  Ex,  the 
barriers  Bcx,  Bwx,  and  the  affinity  it  is  possible  to  “tune” 
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the  system  to  have  any  steady-state  distribution.  However, 
once  the  steady-state  distribution  is  chosen,  the  current  is 
set  by  Eq.  (46)  and  cannot  be  changed.  This  proves  that  the 
constraints  indeed  limit  the  system;  without  them,  any  Jss 
can  be  achieved  with  any  pss  by  tuning  Ew,  E(  .  and  Bcw 
in  Eq.  (46). 

Next,  let  us  consider  the  cost  of  kinetic  proofreading, 
namely,  the  total  entropy  production  rate:  It  has  three 
contributions  from  the  three  edges.  The  currents  in  all  three 
edges  are  identical  and  are  set  by  the  choice  of  steady-state 
probabilities;  thus,  the  differences  between  the  entropy 
production  rates  in  the  different  edges  are  only  due  to  the 
differences  in  the  In(7?.(/ /r“/72;, pf)  factors.  Any  (positive) 
value  of  oscsx  can  be  achieved  by  choosing  the  energy  of  the 
auxiliary  state  to  be 


Ex  =  Ec  -\ — —  +  In  ^-4- , 

x  C  sJSS  nSS  ’ 

Px 


(49) 


as  can  be  verified  by  substituting  the  above  Ex  in  the 
definition  of  the  entropy  production  rate.  Similarly,  it  is 
possible  to  achieve  any  positive  value  of  asfx  by  choosing 
the  affinity  to  be 


</>  =  Ex-Ew-^  +  In^E. 

r  x  W  fjss  1  nss 


(50) 


The  last  contribution  to  the  total  entropy  production  comes 
from  the  cw  edge  and  is  given  by 


Gscsw  —  JSS{EC  —  Ew  +  ln£)NESs),  (51) 

where 

ns s 

dness  =  (52) 

Pw 

is  the  NESS  discrimination  ratio.  In  contrast  to  the  other 
two  edges,  this  entropy  production  rate  is  set  by  the  steady- 
state  distribution  and  the  given  state’s  energies;  hence,  it 
cannot  be  controlled  at  all.  It  is  therefore  a  minimum  bound 
on  the  entropy  production  rate  for  any  NESS  kinetic 
proofreading  to  achieve  the  target  probabilities  in  this 
system.  If  DNESS  =  Deq,  then  lnDNESS  =  Ew  -  Er:  hence, 
the  minimal  entropy  production  is  zero,  as  expected. 

In  this  simple  three-state  system,  any  discrimination  ratio 
DNEss  can,  in  principle,  be  achieved  with  an  arbitrary  low 
cost  in  terms  of  the  total  entropy  production  rate,  by 
decreasing  both  pscs  and  pf  while  keeping  their  ratio 
fixed.  The  price  for  having  low  entropy  production, 
however,  is  that  most  of  the  time,  the  system  is  neither 
in  the  correct  state  c  nor  in  the  incorrect  state  w,  but  in  the 
auxiliary  state  x.  It  is  possible  to  achieve  pscs  arbitrarily 
close  to  1,  but  with  a  logarithmic  price  in  the  discrimination 
ratio  Dness  (even  if  pscs  =  1 ,  Jps  is  finite).  This  trade-off  is 


common  in  NESS  kinetic  proofreading;  see,  e.g., 
Refs.  [41,42], 

Increasing  the  discrimination  ratio  between  two  possible 
outcomes  beyond  its  equilibrium  value  might  have  other 
applications,  for  example,  in  self-assembly.  Many  self- 
assembly  schemes  are  based  on  free-energy  minimization, 
where  the  ratio  of  success  is  dictated  by  the  equilibrium 
distribution.  Controlling  this  ratio,  either  in  NESS  or 
through  periodic  driving,  might  enable  self-assembly  of 
structures  that  cannot  be  achieved  in  equilibrium  [43]. 

B.  Kinetic  proofreading  by  periodic  driving 

1.  Constraints  on  periodic  driving 
for  kinetic  proofreading 

Can  kinetic  proofreading  be  achieved  without  breaking 
detailed  balance  but  through  periodic  driving?  As  we  have 
seen,  any  NESS  system  that  describes  kinetic  proofreading 
can,  in  principle,  be  mapped  into  a  SP  that  mimics  the 
NESS.  However,  in  the  construction  suggested  in  Sec.  V, 
Ec  and  Ew  are  not  constant  as  they  change  throughout  the 
cycle.  This  makes  the  construction  unsuitable  for  kinetic 
proofreading:  A  periodic  driving  kinetic  proofreading  is  a 
time-dependent  detailed-balance  matrix  W(f),  with  the 
additional  constraints 


Ec(t)  =  0,  Ew(t)  =  0,  Bcw(t)  =  0.  (53) 


Without  the  constraints  on  Ec  and  Ew,  it  is  possible  to 
directly  change  the  equilibrium  discrimination  ratio,  and 
thus,  there  is  no  need  for  kinetic  proofreading.  The 
constraint  on  Bcw  is  motivated  by  the  physical  argument 
that  if  the  two  states  c  and  w  are  not  affected  by  the  external 
manipulation,  the  barrier  between  them  should  not  be 
affected  either.  Note  that  these  constraints  do  not  mean 
that  the  nc  and  nw  are  time  independent  since  they  are  also  a 
function  of  the  time-dependent  Ex(t)  through  the  partition 
function  Z.  However,  their  ratio, 


Xc(t) 

nw(t) 


(54) 


is  time  independent.  The  only  physical  parameters  in  W(f) 
that  can  be  manipulated  are  those  of  the  auxiliary  state  x, 
namely,  its  energy  Ex  and  its  banner  heights,  Bwx  and  Bcx. 
Mapping  between  the  kinetic  proofreading  NESS  to  a 
constrained  W(f)  such  that  Eq.  (13)  holds  shows  that  the 
effect  of  any  external  drive  f  can  be  exactly  mimicked  by  a 
time-dependent  auxiliary  state. 

As  in  the  NESS  kinetic  proofreading,  once  the  target 
averaged  probabilities  pfs  are  chosen,  Jps  is  set  by  the 
constraints  in  Eq.  (53): 
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^  0Vcw(t)pV(t)  -  Wwc(t)pps(t))dt 
Jo 

=  Wclv  f  pps(t)dt  —  Wwc  f  pcs(t)dt 

Jo  Jo 

=  e~B™(eEwpw  —  eEcpps).  (55) 

Comparing  with  Eq.  (46),  we  note  that  pfs  =  pfs  imphes 
Jew  =  Jew-  In  other  words,  the  choice  of  distribution,  p\s 
or  pp\  uniquely  defines  the  cotTesponding  currents,  J ,  or 
JPew,  and  a  given  set  of  probabilities  pf  =  pps  gives  rise  to 
the  same  currents  in  both  the  periodic  and  steady-state 
systems. 

In  what  follows,  we  construct  a  periodic  detailed  balance 
rate  matrix  W(f)  with  a  given  Ec,  Ew,  and  Bcw,  such  that  its 
time  averaged  probabilities  can  be  any  (normalized)  prob¬ 
ability  vector  pps,  using  the  tools  developed  in  the  previous 
sections.  For  simplicity,  we  first  show  how  to  construct  the 
protocol  without  considering  the  entropy  production  rates. 
Then,  we  show  that  the  entropy  production  rates  can  be 
controlled  by  tuning  the  free  parameters  in  the  construction 
and  that  the  bound  on  the  minimal  averaged  total  entropy 
production  rate  is  equal  to  the  bound  in  NESS  kinetic 
proofreading. 

2.  Construction  of  a  proofreading  protocol 

We  start  with  a  simple,  physical  intuition  for  the  kinetic 
proofreading  protocol.  As  evident  from  Eq.  (55),  the 
energies  of  the  states  c  and  w,  together  with  the  barrier 
between  them,  as  well  as  the  target  averaged  probabilities, 
dictate  the  averaged  current  that  flows  between  these  states. 
Without  loss  of  generality,  we  assume  that  <5  <  1  but  pps  > 
pps  (the  other  cases  can  be  worked  out  similarly).  Under 
this  assumption,  Jpsc  >  0;  namely,  on  average,  probability 
flows  along  the  wc  edge  from  the  state  c  into  the  state  w. 
However,  because  of  the  no-current-loops  condition,  it  is 
not  possible  to  compensate  for  both  the  probability  that 
flows  out  of  c  and  the  probability  that  flows  into  w  at  each 
instance  of  time  since  the  former  requires  current  from  x 
into  c  and  the  latter  from  w  into  x,  which,  together  with  the 
current  on  the  wc  edge,  form  a  current  loop.  Thus,  as  in 
Sec.  V,  we  divide  the  cycle  time  into  two  halves.  In  the  first 
half-cycle,  the  probability  current  Jacx  that  flows  into  the 
state  c  and  which  is  the  only  probability  current  flowing 
into  the  state  c  over  the  entire  cycle  must  compensate  for  all 
the  probability  currents  that  flow  out  of  the  state  c  during  a 
complete  cycle,  namely,  for  J%c,  J%c-  and  Jaxc  (see  Fig.  6 
for  the  current  pattern  in  the  two  halves  of  the  cycle).  By  the 
no-loop  condition,  during  the  same  half-cycle,  the  current 
along  the  xw  edge  must  flow  from  x  into  w,  further 
increasing  the  accumulated  probability  in  w.  These  currents 
can  be  achieved  by  increasing  the  energy  of  the  auxiliary 
state,  lowering  the  barrier  between  x  and  c  and  increasing 


FIG.  6.  Patters  of  currents  in  periodically  driven  kinetic  proof¬ 
reading.  In  the  first  half-cycle  (left),  both  currents  are  out  of  the 
state  x,  and  in  the  second  half-cycle  (right),  they  are  both  into  the 
state  x. 


the  barrier  between  x  and  w,  to  ensure  that  most  of  the 
probability  from  x  flows  into  c  and  not  w.  In  the  second 
half-cycle,  the  probability  that  accumulates  during  a  cycle 
in  w  is  evacuated  into  x.  To  avoid  a  current  loop,  in  this 
half-cycle  some  of  the  probability  from  c  must  also 
backflow  into  x.  This  is  achieved  by  lowering  the  energy 
of  x,  lowering  the  barrier  between  w  and  x,  and  raising  the 
barrier  between  x  and  c,  to  ensure  that  most  of  the 
probability  that  flows  into  x  comes  from  w,  rather  than 
from  c. 

To  make  this  intuitive  heuristic  into  a  quantitative 
protocol,  we  apply  the  tools  developed  so  far.  This  is  done 
in  three  steps.  In  the  first  step,  we  choose  the  master 
equation’s  solution,  pps(t),  such  that  pps  are  the  target 
probabilities.  In  the  second  step,  we  set  Ex  in  the  two  half¬ 
periods  such  that  the  current  pattern  is  consistent  with  the 
no-current-loops  condition.  In  the  last  step,  we  use  the 
transformation  of  the  symmetric  part  of  the  rate  matrix 
(described  in  Sec.  IV  C)  to  set  Bwx(t)  and  Bcx(t )  such  that 
pps(t)  is  the  solution  of  the  master  equation.  These  steps 
are  detailed  below. 


Step  1:  Choosing  the  solution  p(t). — To  construct  W(f), 
we  first  set  pps(t)  to  be  continuous  and  piecewise-linear  in 
time,  with  opposite  slopes  in  the  two  half-cycles  and  with 
the  target  averaged  probabilities 


PPc\t) 


pPeS-'-f+mt  0  <t<l 

W +  l 'f-mt  \<t<T , 


(56) 


pPw(t) 


pV-f+mt  0  <t<\ 
Pw  +  -  —  mt  p  <  t  <  T. 


(57) 


The  mT/4  and  3inT /4  constants  assure  that  the  average 
probabilities  are  indeed  pps  and  pps.  Note  that  the  slopes  of 
pps{t )  and  pps(t)  are  equal  to  each  other  but  are  opposite  in 
sign  to  the  slope  of  pp\t )  =  1  —  pp\t )  —  pp\t),  which  is 
2m  in  amplitude.  The  slope  m  is  chosen  subject  to  the 
following  constraint: 
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m  >  e  K,:w  max 
o<r<r 


Pw[t) 

PPc\t) 

e~E - 

1 

(58) 


P? (0 ' 

Ppc\t)  e~E ' 


(63) 


where  T  >  0  is  chosen  arbitrarily.  Once  m  is  chosen,  we 
have  to  check  that  all  the  probabilities  are  bounded  away 
from  1  and  0.  If  they  are  not,  we  rechoose  T  to  be  short 
enough  such  that  they  become  bounded  away  from  0  and  1 , 
and  update  Pw(t)  and  pps(t),  but  not  m,  accordingly.  The 
condition  on  m  ensures  that  the  magnitude  of  the  current 
between  states  c  and  w,  given  by 


Pxs{t)  e  E°  _  nax 
Pw(t )  >  e~E "  nw  ' 


(64) 


By  the  assumption  5  <  1  and  pps(t)  >  pw(t),  the  second 
inequality  follows  from  the  first  one.  Using  nw  =  nc8~x 
[from  Eq.  (54)]  and  nw  +  nc  +  nx  =  1  (probability  nor¬ 
malization),  we  can  express 


\JPM\ 


=  e 


Ppw\t) 

PPc\t ) 

e-K 

1 

(59) 


is  smaller  than  the  magnitudes  pps  =  pps  =  ±/«  at  all  t. 
Since  ppcs  =  Jpcx  +  Jpsw  and  similarly  pff  =  Jpjx  +  Jpjc, 
the  signs  of  J cx  and  Jwx  are  always  identical  to  the  signs  of 
pps  and  pps,  and  hence  to  each  other.  Therefore,  Jpcx  and 
Jpjx  are  either  both  into  the  auxiliary  state  x  or  are  both  out 
of  it,  and  there  are  no  current  loops. 

The  condition  on  m  can  also  be  viewed  as  a  condition 
that  assures  that  the  dynamics  is  far  from  the  adiabatic  limit: 
If  the  changes  in  p  are  slow  enough  such  that  the  dynamics 
follows  the  adiabatic  approximation,  then  pps(t)  Kn(t)\ 
thus,  D  «  Deq  and  there  is  no  kinetic  proofreading. 


Step  2:  Setting  Ex(t). — In  the  previous  step,  the  values  of 
pit),  and  therefore  of  the  currents  in  the  system,  were 
chosen.  This  was  done  consistently  with  the  no-current- 
loops  condition.  Next,  we  choose  the  auxiliary  state  energy 
in  the  two  halves  of  the  cycle,  Ex  and  Ex  (as  before, 
superscripts  a  and  b  stand  for  the  first  and  second  half¬ 
cycles,  respectively)  in  a  way  that  is  consistent  with  the 
directions  of  the  currents.  To  this  end,  we  can  express  the 
periodic  state  current  between  the  ith  and  j th  states  as 


Jp°(t)  =  e-W 


(60) 


=  1  ~nc( 1  +5  1)- 


Substituting  this  in  the  above  inequality  gives 


l-^,(l+<5-1)_  _J 

PPc\t)  > 


(1+-5-1). 


(65) 


This  gives  the  exact  condition  for  choosing  nac\ 


1 

1  +8~l 


>  71  r 


>  mfxWib 

PPc\t) 


1 

+  1  +  5-1  ' 


(66) 


where  the  left  inequality  follows  from  the  equality  np\t )  = 
8 ~l7rps(t)  and  probability  normalization.  From  nac  (and 
similarly  for  n)’.),  we  can  solve  for  Ex  (for  Ex).  Using 


o~Ec 


n~K- 


+  e  Ec  +  e  Ew 


■  =  n{ 


,a,b 


(67) 


we  can  extract 


(68) 


A  similar  argument  can  be  applied  to  the  second  half-cycle. 
In  this  case,  the  currents  should  be  from  states  c  and  w  into 
the  state  x\  thus, 


The  direction  of  the  current  Jp-{t)  is  thus  given  by  the  sign 
of  {[PjS{t)\/[e~Ei^]}  —  {[pfs{t)\/[e~E‘^]}.  To  ensure  that 
in  the  first  half-cycle  the  directions  of  the  currents  are 
indeed  from  the  auxiliary  state  x  into  states  c  and  w,  we 
therefore  must  choose  Ex  such  that  both 


PPx{t)  Ppw\t ) 


(69) 

(70) 


and 


(61) 


By  the  assumption  8  <  1  and  pps{t )  >  pps{t),  the  first 
inequality  follows  from  the  second  one,  which  can  be 
written  as 


(62) 


Pxs(t)  <  e_^_  =  1  -^.(1  +8  ‘) 

Pw(t )  e~E»  nw  8~xnc 


(71) 


which  means 


namely, 
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<  mm - 

*  1  +  5" 


(1  + 


Pp:w 


(72) 


Once  nhc  is  chosen,  Eb  can  be  expressed  through  Eq.  (68). 

Note  that  both  Ec‘  and  Ebc  are  not  uniquely  set  but  can  be 
chosen  in  some  range  of  values.  Although  different  choices 
lead  to  the  same  averaged  probabilities  and  current,  their 
averaged  entropy  production  rates  are  different. 


Step  3:  Transforming  the  symmetric  part  of  the  rate 
matrix. — So  far,  the  instantaneous  equilibrium  distribution 
n (f)  and  the  periodic  solution  pps(t )  were  chosen  such  that 
there  are  no  current  loops.  It  is  therefore  possible  to  use  the 
transformation  of  the  symmetric  part  of  the  rate  matrix, 
discussed  in  Sec.  IV  C,  to  adjust  the  magnitudes  of  the 
currents  Jpx(t)  and  J px  ( t)  to  agree  with  p.  In  the  first  half¬ 
cycle, 


q-ps 
U  cx 


=  Pc  -  Jew  —  rn  +  e 


(73) 


3.  Entropic  cost  of  kinetic  proofreading  SP 

To  complete  the  NESS-SP  equivalence  in  this  three-state 
system  kinetic  proofreading  example,  we  should,  in  prin¬ 
ciple,  match  the  entropy  production  rates  of  each  of  the 
edges  in  the  SP  protocol  to  those  of  the  NESS.  While  this  is 
definitely  possible,  for  simplicity,  we  limit  our  discussion 
only  to  the  total  averaged  entropy  production  rate,  which  is 
the  total  cost  of  the  kinetic  proofreading  protocol.  In  what 
follows,  we  show  that  the  total  entropy  production  is 
bounded  from  below  and  that  its  minimal  value  is  the 
same  as  that  of  the  NESS  process.  In  other  words,  NESS 
and  SP  kinetic  proofreading  can  be  achieved  at  the  same 
entropy  production  cost.  To  this  end,  we  first  split  the  total 
entropy  production  rate  as  follows: 


aij  j 


\ 


Jij(t)  In 


Wij(t)PjS(t) 

w jMprit ) 


dt 


J, At)  In 


WyM 


+  J ij(t )  In 


,yw  w,-,-(0  yw  pPi\t) 


dt. 


(77) 


where  the  explicit  time  dependence  is  omitted.  The  con¬ 
dition  on  m  in  Eq.  (58)  ensures  that  both  of  these  currents 
are  indeed  positive.  Moreover,  by  the  current  definition, 


The  first  term,  J0r  J ij{t)  ln{[Wy(f)]/[W;i(f)]}rii,  is  com¬ 
monly  associated  with  the  entropy  production  of  the  heat 
bath  to  which  the  system  is  coupled,  and  the  second  term, 
J  ij{t)\n{[pps  (t)]/\pps  (t)]}dt,  to  the  entropy  produc¬ 
tion  of  the  system  itself.  For  the  periodic  state,  the  sum  of 
all  the  latter  vanishes  [37], 


JPcx(t) 

JPwSx(t) 


=  e 


=  e 


PPc\t) 

PPJ(t)\ 

(75) 

e~Ec 

—  Fa  1  » 

e  J 

' ppA* ) 

ppxsm 

(76) 

e~E„ 

e~E“  j ' 

Combining  Eq.  (75)  with  Eq.  (73),  it  is  straightforward  to 
solve  for  Bxc.(t),  and  similarly  from  Eq.  (76))  with  Eq.  (74) 
to  solve  for  Bxw(t).  The  second  half-cycle  values  of  Bxw  and 
Bxc  can  be  solved  in  the  same  way.  As  expected  from  the 
heuristic  description,  in  the  first  half-cycle,  Bxc{t)  <  Bxw{t) 
and  in  the  second  half-cycle,  Bxc(t )  >  Bxw{t).  Using  these 
Bcx(t)  and  Bwx(t )  together  with  Ex  in  the  first  half-cycle 
and  Eb  in  the  second  half,  we  get,  by  construction,  the 
targeted  probabilities  as  the  averages  of  the  periodic 
solution  pps(t). 

The  construction  for  periodically  driven  kinetic  proof¬ 
reading  protocol  given  above  uses  the  same  tools  as  the 
construction  of  the  NESS-SP  mapping,  with  a  crucial 
difference:  In  the  NESS-SP  mapping,  the  symmetric  part 
of  the  rate  is  kept  fixed  during  each  half-cycle,  and  the 
equilibrium  distribution  changes  with  time;  in  the  kinetic 
proofreading  protocol,  it  is  exactly  the  opposite.  This 
difference  makes  it  simpler  to  construct  a  protocol  that 
respects  the  constraint  on  the  equilibrium  distribution. 


fT  pTi1) 

/  Jo ( o  in  )«7a  = 0- 

U  Jo  Pi  f) 


(78) 


Thus,  the  total  entropy  production  rate  is  given  by 


In 


><j 


t<j 


'M Idt 


=  fT  JiMEj-Eddt. 


(79) 


><J 


As  the  energies  are  fixed  during  each  half-cycle,  this 
expression  can  be  simplified  into 

=  JPc(Ec  -  Ew)  +^JwX(E"  -  Ew) 

ij 

+  \j%c{Eax  -  Ec)  +  ^JL(£w  -  Eb) 

+  l-jfc(Ec-Eb) 

=  m{E“-Eb),  (80) 

where  we  used  Jbw  -  Jawx  =  2JPX„,  Jbxc  -  Jacx  =  -2  Jpx, 
and  as  there  is  only  one  averaged  current  in  the  system. 
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Jxw  =  Jwc  =  Jcx  as  well.  In  addition,  we  used  J '°x  + 
Jacx  —  —px  =  2 m  and  similarly  for  the  second  half¬ 
cycle,  Jbxw  +  Jbxc  =  pb  =  2  in. 

We  can  now  bound  the  minimal  total  entropy  production 
rate  required  to  achieve  a  given  SP  discrimination  factor, 
Ds P,  defined  by 


D  SP  =  PPc/pPJ, 


(81) 


using  the  above  protocol.  We  first  note  that  the  constraint 
on  m  in  Eq.  (58)  implies  that  m  >  Jwc,  where  the  equality 
is  achieved  only  in  the  limit  T  -*  0.  In  addition,  we  can 
bound  Ex  and  Eb,  using  Eqs.  (66),  (68),  and  (72): 

Ex>~ In  (e-E<:  (s~l  +  min ~  e~EA ,  (82) 

Eb  <  -In  (e-t'S-1  ^1  +  max ~  -  (83) 

but  e~EcS~l  =  e~Ew:  thus, 

Ex  >  Ec  -  In  ^min  ,  (84) 

Eb  <  Ew  -  In  (max  ■  (85) 


Substituting  these  bounds  into  the  total  entropy  production 
rate  Eq.  (80)  and  using 


max 


PPc\t) 

Ppw\t) 


>  DSP , 


we  get 


c  >  Jwc{Ec  —  Ew  +  lnDSP).  (86) 


This  bound,  which  can  be  attained  in  the  T  -*■  0  limit  by  a 
proper  choice  of  m,  E"  and  Eb,  is  exactly  the  bound  on 
NESS  kinetic  proofreading  (for  the  same  target  probabil¬ 
ities)  given  in  Eq.  (51).  On  the  other  hand,  the  total 
averaged  entropy  production  rate  can  be  made  arbitrarily 
large  by  choosing  m  to  be  large  enough.  This  shows  that 
any  NESS  kinetic  proofreading  in  this  system,  namely,  any 
value  of  (j>,  Ex,  Bxw,  and  Bxc,  can  be  mimicked  (in  terms  of 
probabilities,  current,  and  total  entropy  production  rate)  by 
the  protocol  described  above. 


VII.  CONCLUSIONS  AND  FUTURE  PERSPECTIVE 

In  this  paper,  we  have  shown  that  the  NESS  and  SP 
paradigms  are  equivalent  in  terms  of  averaged  probabilities, 
currents,  and  entropy  production  rates,  and  we  obtained  an 
explicit  mapping  between  them.  We  then  applied  these 
results  to  construct  a  SP  kinetic-proofreading  protocol  in  a 
three-state  system.  These  results  are  the  first  steps  in 
clarifying  the  precise  relationship  between  these  two 
paradigms,  and  they  were  established  using  a  powerful 


and,  in  our  view,  somewhat  surprising  observation: 
Although  finding  the  average  properties  of  an  arbitrary 
SP  protocol  is  a  difficult  problem  that,  in  general,  can  only 
be  solved  numerically,  the  inverse  problem — finding  a 
time-periodic  protocol  that  has  a  desired  set  of  averaged 
properties — is  an  easy  problem  which  only  requires  solving 
a  set  of  algebraic  linear  equations. 

The  mapping  between  NESS  and  SP  presented  above 
was  not  intended  as  a  set  of  operational  instructions  for 
experimental  realization:  Indeed,  in  most  experimental 
realizations,  not  all  the  rates  can  be  individually  controlled; 
hence,  the  specific  protocol  suggested  above  might  not  be 
applicable.  Rather,  our  intention  is  to  show  that  SP  are  as 
versatile  a  tool  as  NESS  in  controlling  molecular  motors 
and  other  stochastic  processes  and  that  it  is  possible  to 
design  driving  protocols  with  desired  outcomes. 

Biological  systems  suggest  inspiring  examples  of  com¬ 
plicated  molecular  machines  with  different  functionalities 
— from  transport  of  ingredients  by  molecular  motors  to 
kinetic  proofreading  in  biomolecular  synthesis — all  oper¬ 
ating  at  NESS.  Although  it  is  possible  to  break  detailed 
balance  in  the  laboratory  (e.g.,  by  applying  external  voltage 
or  creating  temperature  or  concentration  gradients),  it  is 
difficult  to  design  a  molecular  machine  that,  similar  to 
biological  molecular  machines,  exploits  external  resources 
to  drive  the  system  in  exactly  the  desired  way.  Expanding 
the  tools  at  our  disposal  to  include  periodic  driving  might 
therefore  significantly  increase  our  ability  to  control 
molecular  motors.  The  toolbox  developed  in  this  manu¬ 
script  suggests  “design  principles”  for  periodic  driving  and 
shows  that  tailoring  a  time-periodic  protocol  with  desired 
properties  is  unexpectedly  simple. 

Many  important  theoretical  and  experimental  aspects  of 
the  mapping  were  not  addressed  in  the  current  manuscript. 
On  the  theoretical  side,  these  include  a  comparison  of  the 
fluctuations  around  the  averaged  values  in  the  two  para¬ 
digms,  the  relaxation  rates  to  the  steady  and  periodic  states, 
and  the  construction  of  mappings  that  preserve  properties 
of  interest  other  than  the  ones  we  studied.  As  an  example  of 
a  potentially  important  theoretical  application,  consider  the 
heat,  power,  and  efficiency  in  heat  engines.  Some  heat 
engines,  e.g.,  thermo-electrical  heat  engines,  are  best 
modeled  as  NESS:  The  different  heat  baths  generate  heat 
fluxes  that  violate  detailed  balance.  In  contrast,  recipro¬ 
cating  heat  engines  (e.g.,  the  Carnot,  Otto,  or  Diesel  cycles) 
are  naturally  modeled  as  SP  [44].  Mapping  the  heat,  power, 
and  efficiency  between  these  two  paradigms  (with  addi¬ 
tional  constraints  on  the  temperature)  might  enable  trans¬ 
ferring  many  known  relations  in  NESS  heat  engines  (e.g., 
efficiency  at  maximal  power  [45^-7])  to  SP  heat  engines, 
where  such  general  relations  have  not  been  proven  yet. 

It  will  be  interesting  to  extend  our  analysis  to  other  cases 
as  well:  One  natural  example  is  diffusive  (rather  than 
discrete-state)  stochastic  dynamics.  Diffusive  systems 
evolving  in  time-periodic  potentials  can  act  as  stochastic 
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pumps  [48],  generating  nonvanishing  average  currents, 
provided  they  satisfy  the  conditions  of  a  no-pumping 
theorem  [22,49]  analogous  to  that  for  discrete-state  systems 
[20-24].  These  similarities  suggest  that  periodically  driven 
diffusive  systems  might  be  able  to  mimic  diffusive  non- 
equilibrium  steady  states,  and  vice  versa.  A  different 
possible  extension  is  to  find  an  analogous  mapping  for 
open  quantum  systems,  where  in  addition  to  amplitudes, 
phases  are  of  importance. 

There  are  a  number  of  experimentally  motivated  ques¬ 
tions  for  which  the  tools  we  developed  might  be  useful. 
Examples  include  the  following:  (i)  What  are  the  exact 
constraints  under  which  the  equivalence  still  holds?  In 
kinetic  proofreading,  we  have  provided  a  specific  example 
where  the  equivalence  holds  even  with  partial  control  over 
the  system.  However,  the  minimal  degree  of  control 
required  to  achieve  equivalence  remains  an  open  question, 
(ii)  What  are  the  attainable  values  of  various  properties 
(e.g.,  averaged  currents  or  probability  ratios)  under  a  given 
set  of  experimental  constraints  ?  For  example,  consider  the 
case  in  which  only  the  surrounding  temperature  can  be 
controlled:  Can  significant  changes  in  the  ratio  of  proba¬ 
bilities  be  achieved  by  periodically  changing  the  temper¬ 
ature  only?  (iii)  In  some  experiments,  it  might  be  possible 
both  to  break  detailed  balance  and  to  drive  the  system 
periodically.  Finding  protocols  that  optimize  properties  of 
interest  under  specific  experimental  constraints  then 
becomes  a  natural  question. 
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To  set  <S,  we  choose  the  entropy  production  rate  to  be  1 
along  all  the  edges.  Using  Eq.  (16),  the  matrix  S  is  given  in 
this  case  by 


-14.248 

4.083 

9.0832 

1.082 

4.083 

-6.2469 

1.082 

1.082 

9.0832 

1.0820 

-14.2481 

4.083 

1.082 

1.082 

4.083 

-6.2469 

(A2) 


and  using  Eq.  (14),  7 Z  can  be  calculated,  though  we  will  not 
need  it  in  what  follows. 

Let  us  demonstrate  our  construction  on  this  NESS.  For 
the  first  step,  we  choose 


5  = 


/  -3 

1 

1 
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!  !  !  \ 

-3  1  1 


(A3) 


We  next  note  that  min(/[log  —  log  TZ-^pf]  =  1/3.  If 

we  therefore  choose  qa  such  that  |  log  n"  —  log  ql  I  <  i/6, 
condition  (25)  is  automatically  satisfied.  Any  vector  close 
enough  to  na  will  do.  As  an  example,  we  use 
qa  =  (0.23,0.24,0.26,0.27).  In  the  second  half  of  the 
period,  these  correspond  to  nb  =  (4, 4, 4, 4)  and  qb  = 
(4.3478, 4.1667, 3.8462,  3.7037). 

The  next  step  requires  the  signs  of  the  currents  generated 
by  TZa  =  UdD)- 1  with  the  probability  qa: 


sign(Jr|3)  =  +1,  sign(Jr}4)  =  +1,  sign^^)  =  +1. 

(A4) 


APPENDIX  A:  DETAILS  OF  THE 
SPECIFIC  EXAMPLE 

The  example  we  consider  is  shown  in  Fig.  2.  It  has  four 
states  with  pss  =  (0.1, 0.2, 0.3, 0.4).  The  spanning-tree 
edges  are  the  dashed-dotted  lines  (red),  and  the  funda¬ 
mental  current  edges  are  the  solid  line  lines  (black).  The 
fundamental  currents  were  chosen  such  that  they  form  a 
loop,  and  their  values  are  J3\  —  3,  —  2,  and  J\\  =  1. 

Once  these  currents  are  chosen,  the  currents  for  the 
spanning-tree  edges  are  dictated  by  Kirchoff’s  law — the 
sum  of  currents  in  each  vertex  must  be  zero.  The  corre¬ 
sponding  current  matrix  is 


Note  that  the  direction  of  the  current  in  the  1-4  edge  is  the 
same  as  the  desired  current  orientation,  but  for  the  other  two 
edges,  the  directions  are  different  from  the  desired  currents. 
In  the  second  step,  we  solve  Eq.  (32)  for  cm- ,  which  gives 

a13  =  1.7188,  (A5) 

«l4  =  5.2366,  (A6) 

a34  =  12.2484.  (A7) 

Together  with  the  directions,  the  fundamental  currents  in 
the  two  half-cycles  are  given  by 


Jss  = 


( 


\ 


1  \ 
1 

J\3  =  5.156, 

J\4  =  7.2366, 

(A8) 

0 

-2 

2 

0 

-3 

1 

J\4  =  24.4969, 

(Al) 

V  ;  J\4  =  -  5.2366, 

J\3  =  - 11-156, 

(A9) 

3 

-1 

0 

-2 

j\4  =  -28.4969. 

(A10) 

-1 

-1 

2 

V 

Note  that  the  averages  are 

the  same  as  the  desired  values. 
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In  the  third  step,  we  solve  for  the  spanning-tree  currents, 
by  solving  Eqs.  (34)  and  (35)  for  j\2,  j\2,  and  J They 
are  given  by 

J\2  =  - 25.4965,  J\3  =  13.4933,  (All) 

J\A  =  9.4902,  J2n  =  -21.4965,  (A12) 

^7*23  —  — 11.4933,  J\A  =  -7.4902.  (A13) 

In  the  fourth  step,  we  calculate  S“  and  Sh  using  Eq.  (39). 
Here,  we  present  only  the  upper  part  as  they  are  symmetric: 
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-885.2 
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79.1 

-842.1 
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/ -596.1 
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-682.8 

143.4 

64.7 

-1032.6 

800.2 
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-897.4/ 

(A14) 


(A15) 


In  the  last  step,  we  first  calculate  dtp  on  the  two  half¬ 
cycles  by  proper  summation  of  the  currents:  d,pa  = 
(37.89,  -2.51, 5.85,  -41.22)  and  dtpb  =  (-37.89, 2.51, 
—5.85,41.22),  which,  as  expected,  cancel  each  other. 
These  are  the  slopes  of  the  p(t)’ s  in  the  first  and  second 
halves  of  the  period.  To  keep  0  <  p(t)  <  1,  we  need  to 
choose  T  small  enough,  say,  for  simplicity,  T  =  0.01. 
Using  T ,  we  can  calculate  p"'h(t'),  which  are  the  actual 
probability  distributions  in  the  two  halves.  These  linear 
functions  are  plotted  in  Fig.  4.  Plugging  these  into  Eq.  (40) 
gives  W(f). 


APPENDIX  B:  ZERO  CURRENTS  IN  THE  NESS 

If  Jff  =  0  for  some  edges,  then  clearly  off  =  0  for  the 
same  edges  as  well.  However,  since  of-  ( t)  >  0  identically 
[this  follows  from  Eqs.  (10)  and  (11)],  we  must  have 
of-  ( t)  =  0  for  all  t  in  order  for  the  time-averaged  entropy 
production  rate  of  the  periodic  state  to  be  zero.  This  can 
only  happen  if  the  currents  along  these  edges  are  zero  at  all 
times,  =  0.  A  simple  prescription  to  set  these 

currents  to  zero  is  to  use  the  following  modified  rate  matrix: 

nfJ=  V~\  (Bl) 

where  V  is  the  matrix  with  the  steady  state  of  77  on  its 
diagonal,  Jss  is  the  steady-state  current  matrix  of  77  (but 
with  a  different  factor  in  front  of  it),  and 


J  0  J  ij  =  0  and  i  f  j 

Sli=USij 

with  the  diagonal  elements  of  S  changed  to  make  the  sum 
of  columns  zero.  This  modified  rate  matrix,  in  which  the 
edges  with  zero  steady-state  currents  have  been  “removed” 
(77(J-  =  0  on  these  edges),  has  the  same  steady  state  as  77, 
but  its  steady-state  currents  are  1.5  times  larger  than  those 
of  77.  From  Eq.  (16),  it  also  follows  that  off  =  1 Doff. 

Note  that  forcing  some  of  the  77(/  to  be  zero  might  make 
the  time-dependent  system  nonergodic  since  it  might 
disconnect  some  of  the  states  from  the  others  at  all  times. 
This,  however,  can  be  overcome  by  dividing  the  time 
interval  into  three  equal  intervals  rather  than  two.  In  the  first 
two  parts,  we  repeat  the  construction  as  before,  but  using  77 
instead  of  77.  In  the  last  interval,  we  choose 
W(t)  =  <Sc(nc)-1,  with  Sff  =  1  for  any  i  f  j  and  nc 
the  diagonal  matrix  with  pps(t  =  0)  on  its  diagonal. 
With  this  construction,  there  are  no  currents  in  the  periodic 
solution  during  the  last  time  interval,  the  average  currents 
and  entropy  production  rates  are  the  required  ones,  and  the 
last  interval  ensures  that  the  system  is  ergodic. 


[1]  R.  Zwanzig,  Nonequilibrium  Statistical  Mechanics  (Oxford 
University  Press,  New  York,  2001). 

[2]  C  Jarzynski.  Equalities  and  Inequalities:  Irreversibility  and 
the  Second  Law  of  Thermodynamics  at  the  Nanoscale, 
Annu.  Rev.  Condens.  Matter  Phys.  2,  329  (2011). 

[3]  E.  W.  Montroll  and  K.  E.  Shuler,  Studies  in  Nonequilibrium 
Rate  Processes.  I.  The  Relaxation  of  a  System  of  Harmonic 
Oscillators,  J.  Chem.  Phys.  26,  454  (1957). 

[4]  H.  Ge,  H.  Qian,  and  M.  Qian,  Stochastic  Theory  of  Non¬ 
equilibrium  Steady  States  and  Its  Applications.  Part  I,  Phys. 
Rep.  510,  87  (2012). 

[5]  R.  Dean  Astumian,  Adiabatic  Operation  of  a  Molecular 
Machine,  Proc.  Natl.  Acad.  Sci.  U.S.A.  104,  19715  (2007). 

[6]  N.  A.  Sinitsyn,  The  Stochastic  Pump  Effect  and  Geometric 
Phases  in  Dissipative  and  Stochastic  Systems,  J.  Phys.  A  42, 
193001  (2009). 

[7]  R.  Kubo,  Statistical-Mechanical  Theory  of  Irreversible 
Processes.  I.  General  Theoty  and  Simple  Applications  to 
Magnetic  and  Conduction  Problems,  J.  Phys.  Soc.  Jpn.  12, 
570  (1957). 

[8]  Joe  Howard,  Molecular  Motors:  Structural  Adaptations  to 
Cellular  Functions,  Nature  (London)  389,  561  (1997). 

[9]  A.  B.  Kolomeisky  and  M.  E.  Fisher,  Molecular  Motors:  A 
Theorist’s  Perspective,  Annu.  Rev.  Phys.  Chem.  58,  675 
(2007). 

[10]  K.  Visscher,  M.  J.  Schnitzer,  and  S.  M.  Block,  Single 
Kinesin  Molecules  Studied  with  a  Molecular  Force  Clamp, 
Nature  (London)  400,  184  (1999). 

[11]  M.  E.  Fisher  and  A.  B.  Kolomeisky,  Simple  Mechanochem- 
istry  Describes  the  Dynamics  of  Kinesin  Molecules,  Proc. 
Natl.  Acad.  Sci.  U.S.A.  98,  7748  (2001). 


021022-19 


O.  RAZ,  Y.  SUBA§I,  and  C.  JARZYNSKI 


PHYS.  REV.  X  6,  021022  (2016) 


[12]  R.  Yasuda,  H.  Noji,  K.  Kinosita,  and  M.  Yoshida,  Fl- 
ATPase  Is  a  Highly  Efficient  Molecular  Motor  that  Rotates 
with  Discrete  120  Degrees  Steps,  Cell  93,  1117  (1998). 

[13]  P.  Gaspard  and  E.  Gerritsma,  The  Stochastic  Chemome- 
chanics  of  the  Fl-ATPase  Molecular  Motor,  J.  Theor.  Biol. 
247,  672  (2007). 

[14]  W.  R.  Browne  and  B.L.  Feringa,  Making  Molecular 
Machines  Work,  Nat.  Nanotechnol.  1,  25  (2006). 

[15]  J.  V.  Hernandez,  E.  R.  Kay,  and  D.  A.  Leigh,  A  Reversible 
Synthetic  Rotary  Molecular  Motor,  Science  306,  1532 
(2004). 

[16]  S.  Erbas-Cakmak,  D.  A.  Leigh,  C.  T.  McTeman,  and  A.  L. 
Nussbaumer,  Artificial  Molecular  Machines,  Chem.  Rev. 
115,  10081  (2015). 

[17]  D.  A.  Leigh,  J.  K.  Y.  Wong,  F.  Dehez,  and  F.  Zerbetto, 
Unidirectional  Rotation  in  a  Mechanically  Interlocked 
Molecular  Rotor,  Nature  (London)  424,  174  (2003). 

[18]  N.  A.  Sinitsyn  and  I.  Nemenman,  The  Berry  Phase  and  the 
Pump  Flux  in  Stochastic  Chemical  Kinetics,  Europhys.  Lett. 
77,  58001  (2007). 

[19]  I.  M.  Sokolov,  A  Perturbation  Approach  to  Transport  in 
Discrete  Ratchet  Systems,  J.  Phys.  A  32,  2541  (1999). 

[20]  Dibyendu  Mandal  and  Christopher  Jarzynski,  A  Proof  by 
Graphical  Construction  of  the  No-Pumping  Theorem  of 
Stochastic  Pumps,  J.  Stat.  Mech.  (2011)  P10006. 

[21]  S.  Rahav,  J.  Horowitz,  and  C.  Jarzynski,  Directed  Flow  in 
Nonadiabatic  Stochastic  Pumps,  Phys.  Rev.  Lett.  101. 
140602  (2008). 

[22]  V.  Y.  Chernyak  and  N.  A.  Sinitsyn,  Pumping  Restriction 
Theorem  for  Stochastic  Networks,  Phys.  Rev.  Lett.  101, 
160601  (2008). 

[23]  C.  Maes,  K.  Netocny,  and  S.  R.  Thomas,  General  No-Go 
Condition  for  Stochastic  Pumping,  J.  Chem.  Phys.  132, 
234116  (2010). 

[24]  D.  Mandal,  Unification  and  New  Extensions  of  the  No- 
Pumping  Theorems  of  Stochastic  Pumps,  Europhys.  Lett. 
108,  50001  (2014). 

[25]  S.  Asban  and  S.  Rahav,  No-Pumping  Theorem  for  Many 
Particle  Stochastic  Pumps,  Phys.  Rev.  Lett.  112,  050601 
(2014). 

[26]  J.  Ren,  V.  Y.  Chernyak,  and  N.  A.  Sinitsyn,  Duality  and 
Fluctuation  Relations  for  Statistics  of  Currents  on  Cyclic 
Graphs,  J.  Stat.  Mech.  (2011)  P05011. 

[27]  S.  Rahav,  Extracting  Work  from  Stochastic  Pumps,  J.  Stat. 
Mech.  (2011)  P09020. 

[28]  M.  Esposito  and  J.  M.  R.  Parrondo,  Stochastic  Thermody¬ 
namics  of  Hidden  Pumps,  Phys.  Rev.  E  91,  052114  (2015). 

[29]  R.  Uzdin,  A.  Levy,  and  R.  Kosloff,  Equivalence  of  Quantum 
Heat  Machines,  and  Quantum-Thermodynamic  Signatures, 
Phys.  Rev.  X  5,  031044  (2015). 

[30]  N.  H.  Lindner,  G.  Refael,  and  V.  Galitski,  Floquet  Topo¬ 
logical  Insulator  in  Semiconductor  Quantum  Wells,  Nat. 
Phys.  7,  490  (2011). 

[31]  K.  I.  Seetharam,  C.-E.  Bardyn,  N.  H.  Lindner,  M.  S.  Rudner, 
and  G.  Refael,  Controlled  Population  of  Floquet-Bloch 
States  via  Coupling  to  Bose  and  Fermi  Baths,  Phys.  Rev.  X 
5,  041050  (2015). 


[32]  Y.  H.  Wang,  H.  Steinberg,  P.  Jarillo-Herrero,  and  N.  Gedik, 
Observation  of  Floquet-Bloch  States  on  the  Surface  of  a 
Topological  Insulator,”  Science  342,  453  (2013). 

[33]  M.  C.  Rechtsman,  J.  M.  Zeuner,  Y.  Plotnik,  Y.  Lumer,  D. 
Podolsky,  F.  Dreisow,  S.  Nolte,  M.  Segev,  and  A.  Szameit, 
Photonic  Floquet  Topological  Insulators,  Nature  (London) 
496,  196  (2013). 

[34]  J.  J.  Hopfield,  Kinetic  Proofreading:  A  New  Mechanism  for 
Reducing  Errors  in  Biosynthetic  Processes  Requiring  High 
Specificity,  Proc.  Natl.  Acad.  Sci.  U.S.A.  71,  4135  (1974). 

[35]  J.  Ninio,  Kinetic  Amplification  of  Enzyme  Discrimination, 
Biochimie  57,  587  (1975). 

[36]  J.  Schnakenberg,  Network  Theory  of  Microscopic  and 
Macroscopic  Behavior  of  Master  Equation  Systems,  Rev. 
Mod.  Phys.  48,  571  (1976). 

[37]  U.  Seifert,  Stochastic  Thermodynamics,  Fluctuation  Theo¬ 
rems  and  Molecular  Machines,  Rep.  Prog.  Phys.  75.  126001 
(2012). 

[38]  R.  K.  P.  Zia  and  B.  Schmittmann,  Probability  Currents  as 
Principal  Characteristics  in  the  Statistical  Mechanics  of 
Non-Equilibrium  Steady  States,  J.  Stat.  Mech.  (2007) 
P07012. 

[39]  If  the  coefficient  matrix  in  these  linear  equations  happens  to 
be  degenerate,  then  they  may  not  have  a  unique  solution. 
Generically,  this  degeneracy  can  be  lifted  by  a  small  change 
in  the  choice  of  seed  probabilities,  qaj. 

[40]  D.  Hartich,  A.  C.  Barato,  and  U.  Seifert,  Nonequilibrium 
Sensing  and  Its  Analogy  to  Kinetic  Proofreading,  New  J. 
Phys.  17,  055026  (2015). 

[41]  A.  Murugan,  D.  A.  Huse,  and  S.  Leibler,  Speed,  Dissipation, 
and  Error  in  Kinetic  Proofreading,  Proc.  Natl.  Acad.  Sci. 
U.S.A.  109,  12034  (2012). 

[42]  R.  Rao  and  L.  Peliti,  Thermodynamics  of  Accuracy  in 
Kinetic  Proofreading:  Dissipation  and  Efficiency  Trade- 
Offs,  J.  Stat.  Mech.  (2015)  P06001. 

[43]  M.  Tagliazucchi,  E.  A.  Weiss,  and  I.  Szleifer,  Dissipative 
Self-Assembly  of  Particles  Interacting  through  Time-Oscil¬ 
latory  Potentials,  Proc.  Natl.  Acad.  Sci.  U.S.A.  Ill,  9751 
(2014). 

[44]  K.  Brandner,  K.  Saito,  and  U.  Seifert,  Thermodynamics  of 
Micro-  and  Nano-Systems  Driven  by  Periodic  Temperature 
Variations,  Phys.  Rev.  X  5,  031019  (2015). 

[45]  M.  Esposito,  R.  Kawai,  K.  Lindenberg,  and  C.  Van  den 
Broeck,  Efficiency  at  Maximum  Power  of  Tow- 
Dissipation  Carnot  Engines,  Phys.  Rev.  Lett.  105, 
150603  (2010). 

[46]  C.  Van  den  Broeck,  Thermodynamic  Efficiency  at  Maximum 
Power,  Phys.  Rev.  Lett.  95,  190602  (2005). 

[47]  M.  Esposito,  K.  Lindenberg,  and  C.  Van  den  Broeck, 
Universality  of  Efficiency  at  Maximum  Power,  Phys.  Rev. 
Lett.  102,  130602  (2009). 

[48]  J.  M.  R.  Parrondo,  Reversible  Ratchets  as  Brownian 
Particles  in  an  Adiabatically  Changing  Periodic  Potential, 
Phys.  Rev.  E  57,  7297  (1998). 

[49]  J.  M.  Horowitz  and  C.  Jarzynski,  Exact  Formula  for 
Currents  in  Strongly  Pumped  Diffusive  Systems,  J.  Stat. 
Phys.  136,  917  (2009). 


021022-20 


